-- Logs begin at Thu 2022-06-16 18:01:08 UTC, end at Thu 2022-06-16 19:19:07 UTC. -- Jun 16 18:44:20.023563 dv-cinder-846172-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 16 18:44:20.032679 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 16 18:44:20.035272 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 16 18:44:20 2022] *** Jun 16 18:44:20.035411 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 16 18:44:20.035516 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 16 18:44:20.035626 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: nodename: dv-cinder-846172-1-iscsi Jun 16 18:44:20.035726 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: machine: x86_64 Jun 16 18:44:20.035821 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: clock source: unix Jun 16 18:44:20.035920 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: pcre jit disabled Jun 16 18:44:20.036021 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: detected number of CPU cores: 4 Jun 16 18:44:20.036120 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: current working directory: / Jun 16 18:44:20.036213 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: detected binary path: /usr/bin/uwsgi-core Jun 16 18:44:20.036322 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: your processes number limit is 47680 Jun 16 18:44:20.036425 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: your memory page size is 4096 bytes Jun 16 18:44:20.036526 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: detected max file descriptor number: 2048 Jun 16 18:44:20.036644 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: lock engine: pthread robust mutexes Jun 16 18:44:20.036896 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: thunder lock: enabled Jun 16 18:44:20.037190 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 16 18:44:20.037388 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 16 18:44:20.081405 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: Python main interpreter initialized at 0x55d11f61d9c0 Jun 16 18:44:20.081405 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: python threads support enabled Jun 16 18:44:20.081405 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: your server socket listen backlog is limited to 100 connections Jun 16 18:44:20.081405 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: your mercy for graceful operations on workers is 90 seconds Jun 16 18:44:20.081761 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: mapped 403077 bytes (393 KB) for 2 cores Jun 16 18:44:20.081826 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: *** Operational MODE: preforking *** Jun 16 18:44:20.082114 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: *** uWSGI is running in multiple interpreter mode *** Jun 16 18:44:20.082114 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: spawned uWSGI master process (pid: 80269) Jun 16 18:44:20.082328 dv-cinder-846172-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 16 18:44:20.082918 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: spawned uWSGI worker 1 (pid: 80283, cores: 1) Jun 16 18:44:20.083763 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: spawned uWSGI worker 2 (pid: 80284, cores: 1) Jun 16 18:44:20.083763 dv-cinder-846172-1-iscsi devstack@keystone.service[80269]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 16 18:44:25.635187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 18:44:25.635187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: from cryptography.utils import int_from_bytes Jun 16 18:44:25.636015 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 18:44:25.636015 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: from cryptography.utils import int_from_bytes Jun 16 18:44:25.639110 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 18:44:25.639110 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: from cryptography.utils import int_from_bytes Jun 16 18:44:25.639574 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 18:44:25.639574 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: from cryptography.utils import int_from_bytes Jun 16 18:44:25.686256 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 18:44:25.686460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 18:44:25.686689 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] command line args: [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 18:44:25.686975 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 18:44:25.687158 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ================================================================================ {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 18:44:25.687418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] admin_token = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.687672 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] config_dir = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.687864 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.688051 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] config_source = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.688231 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] debug = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.688431 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.688936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.689028 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 18:44:25.689091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.689154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 18:44:25.689217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] command line args: [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 18:44:25.689284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.689468 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.689547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 18:44:25.689669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ================================================================================ {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 18:44:25.689743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.689867 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_config_append = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.689982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] admin_token = **** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.690086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.690186 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] config_dir = [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.690289 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_dir = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.690390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.690497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_file = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.690633 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] config_source = [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.690717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.690837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] debug = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.690918 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.691039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.691147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.691219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.691332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.691405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.691520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.691601 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.691672 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.691793 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.691897 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.691981 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] list_limit = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.692147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.692236 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] log_config_append = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.692330 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.692410 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.692525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] log_dir = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.692599 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.692717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] log_file = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.692830 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.692917 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.693065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.693137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.693314 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.693390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.693562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.693645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.693771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.693846 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.693968 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.694183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.694320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.70/identity {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.694433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.694516 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] publish_errors = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.694751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.694837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.694988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.695065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.695166 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.695241 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.695414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.695497 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.695650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.695731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.695885 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] standard_threads = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.695961 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.696117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.696194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.696363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.696540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.696669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.70/identity {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.696755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] use_journal = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.696857 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] publish_errors = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.696939 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] use_json = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.697044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] use_stderr = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.697167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.697241 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] use_syslog = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.697310 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.697439 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.697516 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.697695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.697780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.697935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.698012 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.698169 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] standard_threads = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.698248 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.698404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.698481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.698675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.698755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.698856 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.698942 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.699022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] use_journal = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.699151 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.699314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] use_json = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.699396 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.699499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] use_stderr = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.699625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.699709 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] use_syslog = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.699875 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.699957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:44:25.700125 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.700201 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.700360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.700544 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.701210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.701274 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.701417 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.701669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.701743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.701918 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.702070 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.702154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.702322 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.702405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.702589 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.702681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.702854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.702937 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.703109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.703190 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.703362 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.703513 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.703614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.703692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.703853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.703930 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.704101 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.704184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.704336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.704412 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.704569 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.704646 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.704801 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.704888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.705041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.705124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.705291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.705373 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.705542 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.705627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.705773 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.705849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.706004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.706081 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.706252 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.706336 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.706500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.706632 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.706762 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.706839 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.707012 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.707103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.707245 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.707328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.707479 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.707554 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.707714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.707792 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.707944 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.708020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.708176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.708251 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.708408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.708485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.708638 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.708714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.708869 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.708944 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.709116 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.709193 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.709347 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.709425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.709583 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.709659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.709815 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.709893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.710050 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.710125 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.710286 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.710360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.710527 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.710621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.710784 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.710861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.711017 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.711097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.711250 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.711327 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.711480 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.711582 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.711722 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.711823 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.711973 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.712077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.712207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.712308 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.712448 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.712580 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.712710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.712791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.712913 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.712988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.713102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.713184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.713363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.713442 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.713594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.713669 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.713823 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.713902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.714060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.714144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.714294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.714378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.714530 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.714623 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.714802 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.714893 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.715036 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.715116 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.715288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.715373 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.715523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.715607 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.715754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.715830 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.715990 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.716255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.716329 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.716563 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.716650 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.716989 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.717068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.717232 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.717346 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.717469 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.717556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.717710 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.717786 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.717945 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.718102 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.718188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.718355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.718432 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.718679 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.718802 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.718934 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.719025 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.719176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.719263 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.719416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.719493 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.719654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.719740 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.719947 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.720023 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.720184 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.720299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.720422 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.720508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.720654 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.720731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.720885 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.720962 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.721117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.721269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.721345 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.721414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.721572 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.721648 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.721808 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.721892 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.722041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.722129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.722290 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.722368 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.722528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.722634 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.722793 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.722878 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.723040 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.723119 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.723280 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.723357 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.723509 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.external = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.723628 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.723733 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.723819 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.723963 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.724039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.724194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.724280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.724427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.724505 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.password = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.724675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.724796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.token = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.724879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.724953 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.725121 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.725200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.725359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.725435 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.725589 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.725691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.725833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.725919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.726078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.726155 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.726309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.726389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.726546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.726686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.726818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.726897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] auth.external = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.727068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.727148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.727316 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.727396 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.727552 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.727633 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.727796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.727877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] auth.password = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.728029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.728109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] auth.token = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.728272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.728348 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.728510 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.728591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.728756 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.728834 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.728988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.729071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.729240 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.729327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.729490 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.729570 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.729724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.729802 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.729962 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] credential.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.730039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.730211 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.730290 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.730443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.730520 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.730703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.730785 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.730947 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.731025 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.731191 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.731268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.731422 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.731500 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.731757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.731896 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.732067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.732152 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.732325 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.732523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.732609 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.732791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.732875 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.733029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.733105 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.733276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.733363 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.733512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.733590 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.733760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.733837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.734006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.734092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.734242 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.734320 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] federation.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.734476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.734554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.734747 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.734837 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.734979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.735063 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.735216 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.735292 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.735462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.735615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.735697 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.735769 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.735930 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.736008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.736161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.736247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.736390 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.736469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.736634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.736712 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.736879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.736957 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.737118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.737196 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.737366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.737443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.737609 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.737686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.737842 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.737918 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.738079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.738158 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.738327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.738404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.738554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.738642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.738812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.738890 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.739040 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.739124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.739275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.739360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.739514 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.739597 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.739745 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.739825 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.739976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.740054 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.740205 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.740282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.740452 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.740531 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.740696 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.740773 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.740928 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.741004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.741174 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.741254 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.741420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.741497 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.741650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.741728 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.741887 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.741965 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.742113 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.742195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.742347 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.742424 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.742595 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.742676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.742836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.742914 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.743070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.743146 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.743306 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.743385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.743552 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.743629 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.743782 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.743859 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.744019 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.744097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.744249 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.744328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.744487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.744564 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.744732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.744811 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.744966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.745045 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.745195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.745271 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.745428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.745513 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.745656 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.745733 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.745892 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.745968 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.746127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.746206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.746368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.746445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.746623 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.746703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.746872 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.746958 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.747115 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.747191 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.747916 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.748099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.748189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.748446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.748524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.748757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.748835 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.748997 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.749077 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.749229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.749314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.749477 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.749554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.749715 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.749799 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.749956 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.750034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.750189 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.750268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.750432 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.750510 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.750690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.750771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.750939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.751016 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.751175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.751253 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.751411 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.751488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.751646 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.751724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.751879 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.751958 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.752109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.752185 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.752346 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.752424 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.752581 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.752659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.752815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.752892 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.753049 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.753127 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.753291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.753375 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.753528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.753606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.753762 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.753840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.753999 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.754078 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.754244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.754320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.754477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.754555 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.754731 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.754810 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.754968 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.755052 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.755203 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.755279 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.755440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.755520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.755676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.755754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.755911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.755988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.756158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.756236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.756395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.756474 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.756628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.756704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.756869 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.756948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.757102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.757186 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.757338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.757414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.757570 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] role.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.757648 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.757805 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.757887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] role.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.758033 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.758110 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.758279 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.758363 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.758511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.758605 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.758784 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.758900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.759057 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] resource.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.759133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.759260 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.759339 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.759501 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.759578 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.759737 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.759815 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.759979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.760056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.760211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.760289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.760452 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.760530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.760693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.760770 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.760929 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.761005 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.761170 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.761249 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.761401 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] role.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.761578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.761656 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] role.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.761815 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.761894 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.762065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.762143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.762315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.762393 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.762562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.762689 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.762831 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.763018 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.763789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.763869 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.764251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.764332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.764515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.764591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.764771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.764850 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.765008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.765088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.765241 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.765318 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.765482 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.765560 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.765734 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.765812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.765983 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.766061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.766232 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.766392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.766479 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.766654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.766737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.766908 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.766986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.767159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.767249 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.767395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.767462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.767645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.767722 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.767877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.767962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.768131 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.768208 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.768380 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.768457 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.768619 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.768698 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.768853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.768931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.769087 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.769164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.769320 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.769399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.769559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.769635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.769800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.769877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.770298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.770370 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.770537 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.770630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.770800 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.770985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.771076 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.771257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] token.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.771398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.771496 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.771746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.771853 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.771984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.772241 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.772354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.772490 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.772679 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.connection = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.772768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.772934 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.773013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.773188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.773267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.773430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.773514 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.773665 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.773741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.773901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.773980 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.774134 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.774211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.774370 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.774454 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.774676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.774760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.774866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.774942 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.775104 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.775181 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.775351 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.775431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.775593 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.775671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.connection = **** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.775844 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.775931 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.776082 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.776170 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.776328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.776407 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.776561 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.776640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.776796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.776881 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.777031 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.777108 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.777202 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 18:44:25.777281 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.777492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.777741 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.777976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.778220 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.778457 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.778877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.779320 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.779687 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.779926 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.780175 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.780417 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:44:25.780596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 18:44:25.898203 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:25.898391 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:25.898593 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:25.905214 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:25.905446 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:25.905676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:25.910635 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:25.914742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:25.943887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:25.944065 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:25.944139 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:25.944237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.000288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.001050 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.053596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.053820 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.064876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.065292 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.123674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.123911 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.124135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.124356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.124577 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.124797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.131529 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.131762 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.131985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.132202 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.132420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.132635 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.150391 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.150747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.150853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.150939 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.151017 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.151094 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.151158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.151247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.151324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.151399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.151473 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.151604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.151747 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.151838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.164624 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.167854 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.180094 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.180329 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.180550 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.180746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.180836 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.180973 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.181070 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.181181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.181282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.181381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.181466 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.181606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.181821 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.182034 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:44:26.200114 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.200297 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 18:44:26.201280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.201478 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 18:44:26.201565 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.201744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.201879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 18:44:26.201986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.202238 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.202344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.202483 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.202715 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 18:44:26.203162 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.203432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.203698 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.203960 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.213793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.214065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.214330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.214624 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.214977 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.215259 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.215522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.215807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.223912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.225076 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.227379 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.227653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.227928 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.228204 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.228479 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.228732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.228995 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.229119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.229261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.229400 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.229526 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.229684 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.229793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.229964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.230046 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.230246 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.230505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.230795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.231067 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.231334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.231602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.232734 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.258355 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.260021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.260298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.260518 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.262154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.262436 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.266009 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.266415 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.266732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.268528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.268942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.269225 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.275124 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.275404 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.277257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.277541 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.280830 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.281159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.281438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.281715 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.282031 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.282299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.282564 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.282849 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.282932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.283142 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.283425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.283707 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.283984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.284260 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.284531 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.284797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.302129 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.302405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.305042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.305441 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.307224 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.310102 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.310940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.311393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.314276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.314710 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.316442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.319797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.320212 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.324033 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.324138 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.324333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.324615 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.324895 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.325166 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.325523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.328128 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.328427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.328709 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.328989 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.329263 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.329547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.346934 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.347217 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.347480 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.347740 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.350440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.350761 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.351032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.351316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.355972 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.357869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.358229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.359846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.361555 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.361833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.362933 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.363224 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.363499 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.363784 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.364054 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.366458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.366763 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.367172 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.367652 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.368110 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.378219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.378499 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.378801 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.379080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.379449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.379721 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.380048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.382908 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.383305 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.383674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.383886 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.384265 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.384546 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.384825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.398724 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.402188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.404985 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.405989 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.407859 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.408659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.409501 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.409776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.410053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.411999 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.413731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.416071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.416352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.416645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.417501 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.419182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.423411 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.423697 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.424141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.424415 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.424980 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.426599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.429854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.430146 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.430544 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.430842 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.433611 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.433892 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.434169 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.441616 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.441913 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.442205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.443026 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.443303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.443578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.443849 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.444127 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.444506 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.444778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.445140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.445413 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.445680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.446033 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.446300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.446562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.446909 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.450069 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:44:26.450370 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.451348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.451747 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.452043 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.452428 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.452706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.452983 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.453272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.453544 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.454547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.454547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.454547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.454870 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:44:26.489666 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d11f61d9c0 pid: 80284 (default app) Jun 16 18:44:26.493183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8e7d8-d6c2-42be-a521-af7327ca7733 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:26.493489 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8e7d8-d6c2-42be-a521-af7327ca7733 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:26.493766 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8e7d8-d6c2-42be-a521-af7327ca7733 None None] PATH_INFO: `/v3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:26.499586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1/1] 10.222.0.70 () {50 vars in 791 bytes} [Thu Jun 16 18:44:26 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:26.507297 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d11f61d9c0 pid: 80283 (default app) Jun 16 18:44:34.468208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e785a-1306-4fdf-b0f6-4f7340d56c8a None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:34.468766 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e785a-1306-4fdf-b0f6-4f7340d56c8a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:34.468766 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e785a-1306-4fdf-b0f6-4f7340d56c8a None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:34.470093 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 2/2] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:44:34.477660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b0c67-b722-4ebb-95cc-e257f6070456 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:34.478099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b0c67-b722-4ebb-95cc-e257f6070456 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:34.478224 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b0c67-b722-4ebb-95cc-e257f6070456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:34.552705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: INFO dbcounter [None req-221b0c67-b722-4ebb-95cc-e257f6070456 None None] Registered counter for database keystone Jun 16 18:44:34.573907 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG oslo_db.sqlalchemy.engines [None req-221b0c67-b722-4ebb-95cc-e257f6070456 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80283) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 18:44:34.580379 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writer thread running {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 18:44:34.771622 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG passlib.handlers.bcrypt [None req-221b0c67-b722-4ebb-95cc-e257f6070456 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80283) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 18:44:34.771950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG passlib.handlers.bcrypt [None req-221b0c67-b722-4ebb-95cc-e257f6070456 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80283) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 18:44:34.827565 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-221b0c67-b722-4ebb-95cc-e257f6070456 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:34.829187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-221b0c67-b722-4ebb-95cc-e257f6070456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:34.955056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1/3] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 18:44:35.083023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e785a-1306-4fdf-b0f6-4f7340d56c8a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:35.160420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: INFO dbcounter [None req-d42e785a-1306-4fdf-b0f6-4f7340d56c8a None None] Registered counter for database keystone Jun 16 18:44:35.180204 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG oslo_db.sqlalchemy.engines [None req-d42e785a-1306-4fdf-b0f6-4f7340d56c8a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80284) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 18:44:35.187031 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writer thread running {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 18:44:35.308406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e785a-1306-4fdf-b0f6-4f7340d56c8a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:35.314544 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac9233-bc8d-49ca-a6c3-e6e93210612b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:35.315242 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac9233-bc8d-49ca-a6c3-e6e93210612b admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:35.315528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac9233-bc8d-49ca-a6c3-e6e93210612b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:35.315806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac9233-bc8d-49ca-a6c3-e6e93210612b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:35.324794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac9233-bc8d-49ca-a6c3-e6e93210612b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:35.377900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: WARNING keystone.common.rbac_enforcer.enforcer [None req-daac9233-bc8d-49ca-a6c3-e6e93210612b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 18:44:35.378177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac9233-bc8d-49ca-a6c3-e6e93210612b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:35.388973 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 3/4] 10.222.0.70 () {58 vars in 1324 bytes} [Thu Jun 16 18:44:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 18:44:35.395273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b0c67-b722-4ebb-95cc-e257f6070456 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:35.412011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b0c67-b722-4ebb-95cc-e257f6070456 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:35.416511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff177f-5cc5-4d8b-9fe6-28224af49066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:35.417619 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff177f-5cc5-4d8b-9fe6-28224af49066 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:35.417910 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff177f-5cc5-4d8b-9fe6-28224af49066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:35.418191 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff177f-5cc5-4d8b-9fe6-28224af49066 admin admin] PATH_INFO: `/v3/projects/417f1f7349214e82b739864de0303f59` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:35.419810 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff177f-5cc5-4d8b-9fe6-28224af49066 admin admin] RBAC: Authorizing `identity:get_project(project_id=417f1f7349214e82b739864de0303f59)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:35.470612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d3ff177f-5cc5-4d8b-9fe6-28224af49066 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 18:44:35.470958 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff177f-5cc5-4d8b-9fe6-28224af49066 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:35.475529 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 2/5] 10.222.0.70 () {56 vars in 1207 bytes} [Thu Jun 16 18:44:35 2022] GET /identity/v3/projects/417f1f7349214e82b739864de0303f59 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:37.583633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed012ee-c8a1-4222-b50b-6cdcdf28830c None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:37.584170 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed012ee-c8a1-4222-b50b-6cdcdf28830c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:37.584170 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed012ee-c8a1-4222-b50b-6cdcdf28830c None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:37.588327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 4/6] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:44:37.594258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002060e-8765-46ca-9312-43d16162e83a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:37.594663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002060e-8765-46ca-9312-43d16162e83a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:37.594843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002060e-8765-46ca-9312-43d16162e83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:37.614015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e002060e-8765-46ca-9312-43d16162e83a None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:37.615552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e002060e-8765-46ca-9312-43d16162e83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:37.623572 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 3/7] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 18:44:37.645821 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed012ee-c8a1-4222-b50b-6cdcdf28830c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:37.658520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed012ee-c8a1-4222-b50b-6cdcdf28830c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:37.662779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c8371-33ff-4d6b-afd0-93fc3a0b85ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:37.663405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c8371-33ff-4d6b-afd0-93fc3a0b85ce admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:37.663686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c8371-33ff-4d6b-afd0-93fc3a0b85ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:37.663961 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c8371-33ff-4d6b-afd0-93fc3a0b85ce admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:37.672158 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c8371-33ff-4d6b-afd0-93fc3a0b85ce admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:37.673936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c8371-33ff-4d6b-afd0-93fc3a0b85ce admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:37.682521 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 5/8] 10.222.0.70 () {58 vars in 1324 bytes} [Thu Jun 16 18:44:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:44:37.687920 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002060e-8765-46ca-9312-43d16162e83a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:37.696722 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002060e-8765-46ca-9312-43d16162e83a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:37.701167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c563d-b4e6-44b9-b047-ffc7997af853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:37.702018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c563d-b4e6-44b9-b047-ffc7997af853 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:37.702300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c563d-b4e6-44b9-b047-ffc7997af853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:37.702590 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c563d-b4e6-44b9-b047-ffc7997af853 admin admin] PATH_INFO: `/v3/users/7e2576df31cd4791832f5441b6f98ba5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:37.703978 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c563d-b4e6-44b9-b047-ffc7997af853 admin admin] RBAC: Authorizing `identity:get_user(user_id=7e2576df31cd4791832f5441b6f98ba5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:37.705781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c563d-b4e6-44b9-b047-ffc7997af853 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:37.707354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 4/9] 10.222.0.70 () {56 vars in 1198 bytes} [Thu Jun 16 18:44:37 2022] GET /identity/v3/users/7e2576df31cd4791832f5441b6f98ba5 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:39.945786 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d1460-6ecb-481f-8591-66e50e9fb99b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:39.946313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d1460-6ecb-481f-8591-66e50e9fb99b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:39.946395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d1460-6ecb-481f-8591-66e50e9fb99b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:39.947714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 6/10] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:44:39.951508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f632174-704e-4f07-bbe9-f3421a22ff2a None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:39.951924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f632174-704e-4f07-bbe9-f3421a22ff2a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:39.952159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f632174-704e-4f07-bbe9-f3421a22ff2a None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:39.953504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d817de-f786-485e-81e4-3fbf60ed2df8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:39.953632 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 5/11] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:44:39.953794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d817de-f786-485e-81e4-3fbf60ed2df8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:39.954072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d817de-f786-485e-81e4-3fbf60ed2df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:39.958424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db59a8d-f765-4ea2-8862-8cf35a88f9fb None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:39.958752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db59a8d-f765-4ea2-8862-8cf35a88f9fb None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:39.959173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db59a8d-f765-4ea2-8862-8cf35a88f9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:39.976971 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1db59a8d-f765-4ea2-8862-8cf35a88f9fb None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:39.978238 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1db59a8d-f765-4ea2-8862-8cf35a88f9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:39.997763 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG passlib.handlers.bcrypt [None req-82d817de-f786-485e-81e4-3fbf60ed2df8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80284) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 18:44:39.998294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG passlib.handlers.bcrypt [None req-82d817de-f786-485e-81e4-3fbf60ed2df8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80284) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 18:44:40.044900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-82d817de-f786-485e-81e4-3fbf60ed2df8 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:40.046480 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-82d817de-f786-485e-81e4-3fbf60ed2df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:40.060259 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 6/12] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:44:40.083603 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db59a8d-f765-4ea2-8862-8cf35a88f9fb None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:40.106974 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:40.148303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 7/13] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:44:40.170934 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d817de-f786-485e-81e4-3fbf60ed2df8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:40.188312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db59a8d-f765-4ea2-8862-8cf35a88f9fb None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:40.224913 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cf429-466f-4fa4-8f76-a64292a1b201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:40.225637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cf429-466f-4fa4-8f76-a64292a1b201 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:40.225923 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cf429-466f-4fa4-8f76-a64292a1b201 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:40.226196 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cf429-466f-4fa4-8f76-a64292a1b201 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:40.238720 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050cf429-466f-4fa4-8f76-a64292a1b201 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:44:40.250258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-050cf429-466f-4fa4-8f76-a64292a1b201 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:44:40.250258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:44:40.250258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:44:40.250258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:44:40.250258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:44:40.250258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:44:40.250258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:44:40.250258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:44:40.250258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:44:40.250258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:44:40.250258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:44:40.250258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:44:40.250258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:44:40.250258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:44:40.250258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:44:40.250258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:44:40.251285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:44:40.251285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:44:40.251285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:44:40.251285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:44:40.251285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:44:40.251285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:44:40.251285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:44:40.251285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:44:40.251285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:44:40.251285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:44:40.251285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:44:40.251285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:44:40.251285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:44:40.251285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:44:40.251285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:44:40.251285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:44:40.252079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:44:40.252079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:44:40.252079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:44:40.252079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:44:40.252079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:44:40.252079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:44:40.252079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:44:40.252079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:44:40.252079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:44:40.252079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:44:40.252079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:44:40.252079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:44:40.252079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 7/14] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:44:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:44:40.257162 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cf429-466f-4fa4-8f76-a64292a1b201 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:40.272353 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d817de-f786-485e-81e4-3fbf60ed2df8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:40.274687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:40.311531 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b780429-a0b5-4b78-bec3-9eba66542274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:40.312433 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b780429-a0b5-4b78-bec3-9eba66542274 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:40.312724 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b780429-a0b5-4b78-bec3-9eba66542274 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:40.313006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b780429-a0b5-4b78-bec3-9eba66542274 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:40.343456 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:40.362066 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cf429-466f-4fa4-8f76-a64292a1b201 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:40.380428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b780429-a0b5-4b78-bec3-9eba66542274 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:40.381948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b780429-a0b5-4b78-bec3-9eba66542274 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:40.396863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d9146-187d-4596-a6b0-5aa7c2998057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:40.397534 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d9146-187d-4596-a6b0-5aa7c2998057 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:40.397821 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d9146-187d-4596-a6b0-5aa7c2998057 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:40.398103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d9146-187d-4596-a6b0-5aa7c2998057 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:40.398888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d9146-187d-4596-a6b0-5aa7c2998057 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:44:40.399198 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d9146-187d-4596-a6b0-5aa7c2998057 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:40.400580 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d9146-187d-4596-a6b0-5aa7c2998057 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:40.413380 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 8/15] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:44:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:40.419606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d9146-187d-4596-a6b0-5aa7c2998057 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:40.479407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 8/16] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:44:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:44:40.484812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b780429-a0b5-4b78-bec3-9eba66542274 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:40.487517 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:40.520836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d9146-187d-4596-a6b0-5aa7c2998057 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:40.554714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61071282-56cd-4315-a508-93b589f1072a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:40.555584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61071282-56cd-4315-a508-93b589f1072a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:40.555868 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61071282-56cd-4315-a508-93b589f1072a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:40.556143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61071282-56cd-4315-a508-93b589f1072a None admin] PATH_INFO: `/v3/users/7e2576df31cd4791832f5441b6f98ba5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:40.557338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61071282-56cd-4315-a508-93b589f1072a None admin] RBAC: Authorizing `identity:get_user(user_id=7e2576df31cd4791832f5441b6f98ba5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:40.558814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61071282-56cd-4315-a508-93b589f1072a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:40.560957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 9/17] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:44:40 2022] GET /identity/v3/users/7e2576df31cd4791832f5441b6f98ba5 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:44:40.566155 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61071282-56cd-4315-a508-93b589f1072a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:40.579961 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:40.583051 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b780429-a0b5-4b78-bec3-9eba66542274 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:40.619421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd816e-7b7d-4070-be90-81d3ad2b4e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:40.620217 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd816e-7b7d-4070-be90-81d3ad2b4e9a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:40.620498 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd816e-7b7d-4070-be90-81d3ad2b4e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:40.620776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd816e-7b7d-4070-be90-81d3ad2b4e9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:40.638962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd816e-7b7d-4070-be90-81d3ad2b4e9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:40.640455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd816e-7b7d-4070-be90-81d3ad2b4e9a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:40.661394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b1fd816e-7b7d-4070-be90-81d3ad2b4e9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:44:40.661394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:44:40.661394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:44:40.661394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:44:40.661394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:44:40.661394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:44:40.661394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:44:40.661394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:44:40.661394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:44:40.661394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:44:40.661394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:44:40.661394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:44:40.661394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:44:40.661394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:44:40.661394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:44:40.662282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:44:40.662282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:44:40.662282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:44:40.662282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:44:40.662282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:44:40.662282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:44:40.662282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:44:40.662282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:44:40.662282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:44:40.662282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:44:40.662282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:44:40.662282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:44:40.662282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:44:40.662282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:44:40.662282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:44:40.662282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:44:40.662282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:44:40.663497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:44:40.663497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:44:40.663497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:44:40.663497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:44:40.663497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 9/18] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:44:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:44:40.666767 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61071282-56cd-4315-a508-93b589f1072a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:40.668202 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd816e-7b7d-4070-be90-81d3ad2b4e9a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:40.701427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c20b96-3186-4aeb-98ad-27ad8608b82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:40.702222 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c20b96-3186-4aeb-98ad-27ad8608b82e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:40.702511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c20b96-3186-4aeb-98ad-27ad8608b82e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:40.702812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c20b96-3186-4aeb-98ad-27ad8608b82e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:40.704007 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c20b96-3186-4aeb-98ad-27ad8608b82e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:40.705459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c20b96-3186-4aeb-98ad-27ad8608b82e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:40.707091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 10/19] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:44:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:44:40.712687 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c20b96-3186-4aeb-98ad-27ad8608b82e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:40.724063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:40.768957 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd816e-7b7d-4070-be90-81d3ad2b4e9a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:40.794927 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:40.803814 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5162e-a4ce-49c4-8504-b6ef889c5a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:40.804563 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5162e-a4ce-49c4-8504-b6ef889c5a91 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:40.804866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5162e-a4ce-49c4-8504-b6ef889c5a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:40.805148 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5162e-a4ce-49c4-8504-b6ef889c5a91 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:40.805878 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5162e-a4ce-49c4-8504-b6ef889c5a91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:44:40.806189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5162e-a4ce-49c4-8504-b6ef889c5a91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:40.807638 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5162e-a4ce-49c4-8504-b6ef889c5a91 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:40.814730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c20b96-3186-4aeb-98ad-27ad8608b82e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:40.825773 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 10/20] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:44:40.855631 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d1a9f-20e3-49b5-b804-f0458f3927f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:40.857271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d1a9f-20e3-49b5-b804-f0458f3927f3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:40.857582 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d1a9f-20e3-49b5-b804-f0458f3927f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:40.857866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d1a9f-20e3-49b5-b804-f0458f3927f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:40.858759 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d1a9f-20e3-49b5-b804-f0458f3927f3 None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.domain.id=default, user.id=7e2576df31cd4791832f5441b6f98ba5) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:44:40.859078 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d1a9f-20e3-49b5-b804-f0458f3927f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:40.860563 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d1a9f-20e3-49b5-b804-f0458f3927f3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:40.868682 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 11/21] 10.222.0.70 () {56 vars in 1322 bytes} [Thu Jun 16 18:44:40 2022] GET /identity/v3/role_assignments?user.id=7e2576df31cd4791832f5441b6f98ba5&scope.domain.id=default&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:44:43.627982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa7b94-2be6-4383-8e3f-795805f705b2 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:43.628592 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa7b94-2be6-4383-8e3f-795805f705b2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:43.628592 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa7b94-2be6-4383-8e3f-795805f705b2 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:43.629909 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 11/22] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:44:43.635173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f90185-5242-4448-aaf9-83fa7901eebc None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:43.635568 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f90185-5242-4448-aaf9-83fa7901eebc None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:43.635748 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f90185-5242-4448-aaf9-83fa7901eebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:43.663856 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-22f90185-5242-4448-aaf9-83fa7901eebc None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:43.665116 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-22f90185-5242-4448-aaf9-83fa7901eebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:43.703879 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1d2c5-14e4-4888-bd2f-9f91e0927d9a None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:43.704175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1d2c5-14e4-4888-bd2f-9f91e0927d9a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:43.704444 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1d2c5-14e4-4888-bd2f-9f91e0927d9a None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:43.705809 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 12/23] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:44:43.711133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674a582-c710-4340-affe-2181645e07ab None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:43.711420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674a582-c710-4340-affe-2181645e07ab None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:43.711700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674a582-c710-4340-affe-2181645e07ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:43.741536 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c674a582-c710-4340-affe-2181645e07ab None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:43.743287 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c674a582-c710-4340-affe-2181645e07ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:43.796757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 12/24] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:44:43.798329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcd064-9d6e-492c-be38-592a36a8ef68 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:43.799538 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcd064-9d6e-492c-be38-592a36a8ef68 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:43.799964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcd064-9d6e-492c-be38-592a36a8ef68 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:43.803640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 13/25] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:44:43.810274 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534c239-8781-490c-9617-8c0eba1561e8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:43.810757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534c239-8781-490c-9617-8c0eba1561e8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:43.811182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534c239-8781-490c-9617-8c0eba1561e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:43.840225 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b534c239-8781-490c-9617-8c0eba1561e8 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:43.841654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b534c239-8781-490c-9617-8c0eba1561e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:43.881547 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 13/26] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:44:43.882604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674a582-c710-4340-affe-2181645e07ab None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:43.980020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 14/27] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:44:43.982611 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534c239-8781-490c-9617-8c0eba1561e8 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:43.983097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:43.987389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:44.049414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674a582-c710-4340-affe-2181645e07ab None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:44.086958 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d09692-84a5-4735-bc79-1945ef8844fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:44.087985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d09692-84a5-4735-bc79-1945ef8844fe None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:44.088497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d09692-84a5-4735-bc79-1945ef8844fe None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:44.088985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d09692-84a5-4735-bc79-1945ef8844fe None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:44.089947 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d09692-84a5-4735-bc79-1945ef8844fe None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:44.092295 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d09692-84a5-4735-bc79-1945ef8844fe None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:44.173921 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534c239-8781-490c-9617-8c0eba1561e8 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:44.209649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8782c-c337-41c1-80f9-75e48527f50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:44.210709 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8782c-c337-41c1-80f9-75e48527f50f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:44.211157 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8782c-c337-41c1-80f9-75e48527f50f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:44.211580 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8782c-c337-41c1-80f9-75e48527f50f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:44.227474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8782c-c337-41c1-80f9-75e48527f50f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:44:44.235840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-91f8782c-c337-41c1-80f9-75e48527f50f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:44:44.235840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:44:44.235840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:44:44.235840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:44:44.235840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:44:44.235840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:44:44.235840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:44:44.235840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:44:44.235840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:44:44.235840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:44:44.235840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:44:44.235840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:44:44.235840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:44:44.235840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:44:44.235840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:44:44.235840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:44:44.236691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:44:44.236691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:44:44.236691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:44:44.236691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:44:44.236691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:44:44.236691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:44:44.236691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:44:44.236691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:44:44.236691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:44:44.236691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:44:44.236691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:44:44.236691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:44:44.236691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:44:44.236691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:44:44.236691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:44:44.236691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:44:44.237444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:44:44.237444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:44:44.237444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:44:44.237444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:44:44.237444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:44:44.237444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:44:44.237444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:44:44.237444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:44:44.237444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:44:44.237444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:44:44.237444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:44:44.237444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:44:44.239701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 14/28] 10.222.0.70 () {60 vars in 1131 bytes} [Thu Jun 16 18:44:43 2022] POST /identity/v3/roles => generated 219 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:44:44.240023 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 15/29] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:44:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:44:44.240811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d09692-84a5-4735-bc79-1945ef8844fe None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:44.241848 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ff97f-bfe3-4258-bb1f-14f213a50752 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:44.242369 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ff97f-bfe3-4258-bb1f-14f213a50752 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:44.242895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ff97f-bfe3-4258-bb1f-14f213a50752 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:44.244818 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 16/30] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:44:44.245724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ff97f-bfe3-4258-bb1f-14f213a50752 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:44.397637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:44.411157 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d09692-84a5-4735-bc79-1945ef8844fe None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:44.427032 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:44.450907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777aa17e-1427-4794-8e46-a1ef63844516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:44.451932 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777aa17e-1427-4794-8e46-a1ef63844516 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:44.452461 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777aa17e-1427-4794-8e46-a1ef63844516 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:44.452926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777aa17e-1427-4794-8e46-a1ef63844516 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:44.454024 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777aa17e-1427-4794-8e46-a1ef63844516 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:44.455797 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777aa17e-1427-4794-8e46-a1ef63844516 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:44.467003 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ff97f-bfe3-4258-bb1f-14f213a50752 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:44.479787 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 15/31] 10.222.0.70 () {60 vars in 1131 bytes} [Thu Jun 16 18:44:44 2022] POST /identity/v3/roles => generated 225 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:44:44.482324 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3ba71-0a46-4bbc-aac5-36627d37fa16 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:44.482923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3ba71-0a46-4bbc-aac5-36627d37fa16 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:44.483421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3ba71-0a46-4bbc-aac5-36627d37fa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:44.506365 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-48b3ba71-0a46-4bbc-aac5-36627d37fa16 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:44.508812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-48b3ba71-0a46-4bbc-aac5-36627d37fa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:44.535026 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee9be5-bcba-4e38-a7cc-046a70381673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:44.535715 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee9be5-bcba-4e38-a7cc-046a70381673 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:44.536001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee9be5-bcba-4e38-a7cc-046a70381673 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:44.536279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee9be5-bcba-4e38-a7cc-046a70381673 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:44.537039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee9be5-bcba-4e38-a7cc-046a70381673 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:44:44.537349 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee9be5-bcba-4e38-a7cc-046a70381673 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:44.538868 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee9be5-bcba-4e38-a7cc-046a70381673 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:44.551446 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 17/32] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:44:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:44.557503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee9be5-bcba-4e38-a7cc-046a70381673 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:44.604864 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 16/33] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:44:44.633473 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3ba71-0a46-4bbc-aac5-36627d37fa16 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:44.756913 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee9be5-bcba-4e38-a7cc-046a70381673 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:44.789957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:44.800534 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e8411-3ff4-48db-9ff1-2e5a09361256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:44.801419 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e8411-3ff4-48db-9ff1-2e5a09361256 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:44.802000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e8411-3ff4-48db-9ff1-2e5a09361256 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:44.802280 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e8411-3ff4-48db-9ff1-2e5a09361256 None admin] PATH_INFO: `/v3/users/7e2576df31cd4791832f5441b6f98ba5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:44.803532 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e8411-3ff4-48db-9ff1-2e5a09361256 None admin] RBAC: Authorizing `identity:get_user(user_id=7e2576df31cd4791832f5441b6f98ba5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:44.804983 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e8411-3ff4-48db-9ff1-2e5a09361256 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:44.807216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:44.809242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 18/34] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:44:44 2022] GET /identity/v3/users/7e2576df31cd4791832f5441b6f98ba5 => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:44.809379 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70695cb9-a544-4f89-a002-cf1769e1cfdf None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:44.809853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70695cb9-a544-4f89-a002-cf1769e1cfdf None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:44.810293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70695cb9-a544-4f89-a002-cf1769e1cfdf None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:44.811800 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 19/35] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:44:44.812752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70695cb9-a544-4f89-a002-cf1769e1cfdf None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:44.815062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3ba71-0a46-4bbc-aac5-36627d37fa16 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:44.865970 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9a29a-f881-4f3a-99f8-8eec60059d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:44.866808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9a29a-f881-4f3a-99f8-8eec60059d21 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:44.867258 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9a29a-f881-4f3a-99f8-8eec60059d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:44.867715 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9a29a-f881-4f3a-99f8-8eec60059d21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:44.883259 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9a29a-f881-4f3a-99f8-8eec60059d21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:44.885874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9a29a-f881-4f3a-99f8-8eec60059d21 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:44.915063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-92b9a29a-f881-4f3a-99f8-8eec60059d21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:44:44.915063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:44:44.915063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:44:44.915063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:44:44.915063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:44:44.915063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:44:44.915063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:44:44.915063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:44:44.915063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:44:44.915063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:44:44.915063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:44:44.915063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:44:44.915063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:44:44.915063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:44:44.915063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:44:44.915988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:44:44.915988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:44:44.915988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:44:44.915988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:44:44.915988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:44:44.915988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:44:44.915988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:44:44.915988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:44:44.915988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:44:44.915988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:44:44.915988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:44:44.915988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:44:44.915988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:44:44.915988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:44:44.915988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:44:44.915988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:44:44.915988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:44:44.916795 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:44:44.916795 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:44:44.916795 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:44:44.916795 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:44:44.916795 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 17/36] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:44:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:44:44.917911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ba636-a127-45b0-9d46-74258155b41d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:44.918437 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ba636-a127-45b0-9d46-74258155b41d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:44.918933 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ba636-a127-45b0-9d46-74258155b41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:44.941673 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e52ba636-a127-45b0-9d46-74258155b41d None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:44.947889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e52ba636-a127-45b0-9d46-74258155b41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:44.952566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70695cb9-a544-4f89-a002-cf1769e1cfdf None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:44.991395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7bb791-3484-4f5c-a464-d949379a2855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:44.992195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bb791-3484-4f5c-a464-d949379a2855 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:44.992538 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bb791-3484-4f5c-a464-d949379a2855 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:44.992825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bb791-3484-4f5c-a464-d949379a2855 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:44.994016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7bb791-3484-4f5c-a464-d949379a2855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:44.995496 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7bb791-3484-4f5c-a464-d949379a2855 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:44.997677 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 20/37] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:44:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:44.998524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7bb791-3484-4f5c-a464-d949379a2855 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:45.024529 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 18/38] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:44:45.025681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa56b9-7b54-417f-a03d-536eecbaeaaf None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:45.025973 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa56b9-7b54-417f-a03d-536eecbaeaaf None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:45.026257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa56b9-7b54-417f-a03d-536eecbaeaaf None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:45.027569 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 19/39] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:44:45.028421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa56b9-7b54-417f-a03d-536eecbaeaaf None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:45.069572 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:45.109628 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7bb791-3484-4f5c-a464-d949379a2855 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:45.110223 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:45.134971 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa56b9-7b54-417f-a03d-536eecbaeaaf None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:45.145263 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1d522-ff6f-484d-b243-e2aa9037cb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:45.145929 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1d522-ff6f-484d-b243-e2aa9037cb86 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:45.146215 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1d522-ff6f-484d-b243-e2aa9037cb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:45.146498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1d522-ff6f-484d-b243-e2aa9037cb86 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:45.147258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1d522-ff6f-484d-b243-e2aa9037cb86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:44:45.147570 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1d522-ff6f-484d-b243-e2aa9037cb86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:45.148954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1d522-ff6f-484d-b243-e2aa9037cb86 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:45.166590 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 21/40] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:45.167896 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb9e57-7b8e-4717-8e6c-fc7a3c3dd4fe None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:45.168308 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb9e57-7b8e-4717-8e6c-fc7a3c3dd4fe None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:45.168540 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb9e57-7b8e-4717-8e6c-fc7a3c3dd4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:45.174799 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c798376-ae1a-426f-ad21-43fe47a27095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:45.175688 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c798376-ae1a-426f-ad21-43fe47a27095 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:45.175988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c798376-ae1a-426f-ad21-43fe47a27095 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:45.176267 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c798376-ae1a-426f-ad21-43fe47a27095 None admin] PATH_INFO: `/v3/domains/default/users/7e2576df31cd4791832f5441b6f98ba5/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:45.178721 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c798376-ae1a-426f-ad21-43fe47a27095 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7e2576df31cd4791832f5441b6f98ba5, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:45.180185 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c798376-ae1a-426f-ad21-43fe47a27095 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:45.186485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-92eb9e57-7b8e-4717-8e6c-fc7a3c3dd4fe None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:45.187883 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-92eb9e57-7b8e-4717-8e6c-fc7a3c3dd4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:45.194938 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 20/41] 10.222.0.70 () {58 vars in 1361 bytes} [Thu Jun 16 18:44:45 2022] PUT /identity/v3/domains/default/users/7e2576df31cd4791832f5441b6f98ba5/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:44:45.206800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c798376-ae1a-426f-ad21-43fe47a27095 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:45.275806 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 22/42] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:44:45.276605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb9e57-7b8e-4717-8e6c-fc7a3c3dd4fe None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:45.347240 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:45.406843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb9e57-7b8e-4717-8e6c-fc7a3c3dd4fe None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:45.442142 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec7ec1-0b41-4afd-a8f6-8eae851b6e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:45.442843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec7ec1-0b41-4afd-a8f6-8eae851b6e93 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:45.443128 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec7ec1-0b41-4afd-a8f6-8eae851b6e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:45.443407 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec7ec1-0b41-4afd-a8f6-8eae851b6e93 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:45.444116 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec7ec1-0b41-4afd-a8f6-8eae851b6e93 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:45.445576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec7ec1-0b41-4afd-a8f6-8eae851b6e93 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:45.452160 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c798376-ae1a-426f-ad21-43fe47a27095 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:45.493817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc4083-a9bd-4a24-82f6-3c50fee20cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:45.494512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc4083-a9bd-4a24-82f6-3c50fee20cc0 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:45.494840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc4083-a9bd-4a24-82f6-3c50fee20cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:45.495238 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc4083-a9bd-4a24-82f6-3c50fee20cc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:45.496020 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc4083-a9bd-4a24-82f6-3c50fee20cc0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:45.498270 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc4083-a9bd-4a24-82f6-3c50fee20cc0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:45.507652 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:45.516684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 21/43] 10.222.0.70 () {60 vars in 1131 bytes} [Thu Jun 16 18:44:45 2022] POST /identity/v3/roles => generated 223 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:44:45.528658 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc4083-a9bd-4a24-82f6-3c50fee20cc0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:45.674922 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc4083-a9bd-4a24-82f6-3c50fee20cc0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:45.701640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 23/44] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Jun 16 18:44:45 2022] POST /identity/v3/projects => generated 301 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:44:45.719925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864fcf1-1bcb-4427-aea9-6fb0d7c994f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:45.720763 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864fcf1-1bcb-4427-aea9-6fb0d7c994f1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:45.721052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864fcf1-1bcb-4427-aea9-6fb0d7c994f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:45.721333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864fcf1-1bcb-4427-aea9-6fb0d7c994f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:45.722691 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864fcf1-1bcb-4427-aea9-6fb0d7c994f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:45.724186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864fcf1-1bcb-4427-aea9-6fb0d7c994f1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:45.726292 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 22/45] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:44:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:45.733905 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec7ec1-0b41-4afd-a8f6-8eae851b6e93 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:45.855432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec7ec1-0b41-4afd-a8f6-8eae851b6e93 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:45.859668 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:45.896866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402cd8b-21da-4641-b2eb-09bb637dd3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:45.897543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402cd8b-21da-4641-b2eb-09bb637dd3aa None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:45.897831 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402cd8b-21da-4641-b2eb-09bb637dd3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:45.898113 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402cd8b-21da-4641-b2eb-09bb637dd3aa None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:45.898855 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402cd8b-21da-4641-b2eb-09bb637dd3aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:45.900455 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402cd8b-21da-4641-b2eb-09bb637dd3aa None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:45.928147 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 24/46] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Jun 16 18:44:45 2022] POST /identity/v3/projects => generated 312 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:44:47.421033 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f33a6-9370-44fd-9031-7ddf17994aa6 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:47.421539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f33a6-9370-44fd-9031-7ddf17994aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:47.421539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f33a6-9370-44fd-9031-7ddf17994aa6 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:47.422904 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 23/47] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:44:47.428273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a90c00-1525-494e-8509-09399d4ea585 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:47.428688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a90c00-1525-494e-8509-09399d4ea585 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:47.428971 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a90c00-1525-494e-8509-09399d4ea585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:47.445912 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-68a90c00-1525-494e-8509-09399d4ea585 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:47.447449 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-68a90c00-1525-494e-8509-09399d4ea585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:47.518565 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 25/48] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:44:47.542516 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f33a6-9370-44fd-9031-7ddf17994aa6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:47.611408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:47.649716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f33a6-9370-44fd-9031-7ddf17994aa6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:47.685406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96325de3-0f60-4db9-baab-3b54eda00269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:47.686200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96325de3-0f60-4db9-baab-3b54eda00269 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:47.686486 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96325de3-0f60-4db9-baab-3b54eda00269 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:47.686792 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96325de3-0f60-4db9-baab-3b54eda00269 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:47.698312 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96325de3-0f60-4db9-baab-3b54eda00269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:44:47.707760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-96325de3-0f60-4db9-baab-3b54eda00269 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:44:47.707760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:44:47.707760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:44:47.707760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:44:47.707760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:44:47.707760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:44:47.707760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:44:47.707760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:44:47.707760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:44:47.707760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:44:47.707760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:44:47.707760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:44:47.707760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:44:47.707760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:44:47.707760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:44:47.707760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:44:47.708577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:44:47.708577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:44:47.708577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:44:47.708577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:44:47.708577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:44:47.708577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:44:47.708577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:44:47.708577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:44:47.708577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:44:47.708577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:44:47.708577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:44:47.708577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:44:47.708577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:44:47.708577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:44:47.708577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:44:47.708577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:44:47.709333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:44:47.709333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:44:47.709333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:44:47.709333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:44:47.709333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:44:47.709333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:44:47.709333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:44:47.709333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:44:47.709333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:44:47.709333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:44:47.709333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:44:47.709333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:44:47.709333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 24/49] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:44:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:44:47.715548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a90c00-1525-494e-8509-09399d4ea585 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:47.799738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:47.815393 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a90c00-1525-494e-8509-09399d4ea585 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:47.850210 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882358d5-9a12-432e-ad84-4749f245a026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:47.850913 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882358d5-9a12-432e-ad84-4749f245a026 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:47.851204 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882358d5-9a12-432e-ad84-4749f245a026 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:47.851490 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882358d5-9a12-432e-ad84-4749f245a026 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:47.852247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882358d5-9a12-432e-ad84-4749f245a026 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:44:47.852586 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882358d5-9a12-432e-ad84-4749f245a026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:47.853976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882358d5-9a12-432e-ad84-4749f245a026 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:47.862291 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 26/50] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:44:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:47.867864 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96325de3-0f60-4db9-baab-3b54eda00269 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:47.968989 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96325de3-0f60-4db9-baab-3b54eda00269 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:47.994012 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:48.003039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e9826-266b-4c1c-af00-fe73a2cd9013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:48.003912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e9826-266b-4c1c-af00-fe73a2cd9013 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:48.004198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e9826-266b-4c1c-af00-fe73a2cd9013 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:48.004498 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e9826-266b-4c1c-af00-fe73a2cd9013 None admin] PATH_INFO: `/v3/users/7e2576df31cd4791832f5441b6f98ba5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:48.005725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e9826-266b-4c1c-af00-fe73a2cd9013 None admin] RBAC: Authorizing `identity:get_user(user_id=7e2576df31cd4791832f5441b6f98ba5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:48.007230 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e9826-266b-4c1c-af00-fe73a2cd9013 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:48.008849 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 25/51] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:44:47 2022] GET /identity/v3/users/7e2576df31cd4791832f5441b6f98ba5 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:44:48.014443 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882358d5-9a12-432e-ad84-4749f245a026 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:48.095730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc74d8-d1ed-496e-ad92-ebb9fc2a972b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:48.096019 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc74d8-d1ed-496e-ad92-ebb9fc2a972b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:48.096302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc74d8-d1ed-496e-ad92-ebb9fc2a972b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:48.097674 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 26/52] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:44:48.102678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16a194-7ab7-41f1-8311-66e953266cdc None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:48.102972 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16a194-7ab7-41f1-8311-66e953266cdc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:48.103261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16a194-7ab7-41f1-8311-66e953266cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:48.120822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3c16a194-7ab7-41f1-8311-66e953266cdc None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:48.122286 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882358d5-9a12-432e-ad84-4749f245a026 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:48.122559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3c16a194-7ab7-41f1-8311-66e953266cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:48.156553 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afa515-f2dc-4b58-ba42-6b1f6ac07e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:48.157348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afa515-f2dc-4b58-ba42-6b1f6ac07e90 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:48.157641 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afa515-f2dc-4b58-ba42-6b1f6ac07e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:48.157924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afa515-f2dc-4b58-ba42-6b1f6ac07e90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:48.159314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afa515-f2dc-4b58-ba42-6b1f6ac07e90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:48.160827 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afa515-f2dc-4b58-ba42-6b1f6ac07e90 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:48.162483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 27/53] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:44:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:48.168492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afa515-f2dc-4b58-ba42-6b1f6ac07e90 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:48.187435 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:48.191994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 27/54] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:44:48.215550 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16a194-7ab7-41f1-8311-66e953266cdc None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:48.271213 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afa515-f2dc-4b58-ba42-6b1f6ac07e90 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:48.307168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ab5df-863b-49d7-8b02-6634e3495104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:48.307812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ab5df-863b-49d7-8b02-6634e3495104 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:48.308099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ab5df-863b-49d7-8b02-6634e3495104 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:48.308371 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ab5df-863b-49d7-8b02-6634e3495104 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:48.309241 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ab5df-863b-49d7-8b02-6634e3495104 None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.domain.id=default, user.id=7e2576df31cd4791832f5441b6f98ba5) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:44:48.309547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ab5df-863b-49d7-8b02-6634e3495104 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:48.311006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ab5df-863b-49d7-8b02-6634e3495104 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:48.316870 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 28/55] 10.222.0.70 () {56 vars in 1322 bytes} [Thu Jun 16 18:44:48 2022] GET /identity/v3/role_assignments?user.id=7e2576df31cd4791832f5441b6f98ba5&scope.domain.id=default&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:44:48.317287 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16a194-7ab7-41f1-8311-66e953266cdc None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:48.348055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:48.360839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b6690-f920-468e-b960-ded8ce1595f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:48.361660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b6690-f920-468e-b960-ded8ce1595f4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:48.361950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b6690-f920-468e-b960-ded8ce1595f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:48.362234 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b6690-f920-468e-b960-ded8ce1595f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:48.363497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b6690-f920-468e-b960-ded8ce1595f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:48.364984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b6690-f920-468e-b960-ded8ce1595f4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:48.366724 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 28/56] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:44:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:44:48.372765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ab5df-863b-49d7-8b02-6634e3495104 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:48.459458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:48.481201 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ab5df-863b-49d7-8b02-6634e3495104 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:48.515447 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1501a4f-1813-4e5b-a6c6-0dbf1e8948e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:48.516122 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1501a4f-1813-4e5b-a6c6-0dbf1e8948e5 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:48.516412 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1501a4f-1813-4e5b-a6c6-0dbf1e8948e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:48.516699 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1501a4f-1813-4e5b-a6c6-0dbf1e8948e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:48.517413 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1501a4f-1813-4e5b-a6c6-0dbf1e8948e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:48.518909 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1501a4f-1813-4e5b-a6c6-0dbf1e8948e5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:48.546053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 29/57] 10.222.0.70 () {60 vars in 1140 bytes} [Thu Jun 16 18:44:48 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:44:50.695736 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe36dab-c16d-44a5-b160-a506921256bb None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:50.696283 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe36dab-c16d-44a5-b160-a506921256bb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:50.696283 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe36dab-c16d-44a5-b160-a506921256bb None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:50.697795 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 29/58] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:44:50.704157 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd1cc1-6886-48c0-b7c2-627be8a3a86b None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:50.704605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd1cc1-6886-48c0-b7c2-627be8a3a86b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:50.704717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd1cc1-6886-48c0-b7c2-627be8a3a86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:50.721263 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-38dd1cc1-6886-48c0-b7c2-627be8a3a86b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:50.722460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-38dd1cc1-6886-48c0-b7c2-627be8a3a86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:50.790844 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 30/59] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:44:50.816856 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe36dab-c16d-44a5-b160-a506921256bb None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:50.914997 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe36dab-c16d-44a5-b160-a506921256bb None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:50.948330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed6765-48d1-421b-b589-f225840245a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:50.949281 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed6765-48d1-421b-b589-f225840245a3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:50.949570 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed6765-48d1-421b-b589-f225840245a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:50.949887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed6765-48d1-421b-b589-f225840245a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:50.951082 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed6765-48d1-421b-b589-f225840245a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:50.952541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed6765-48d1-421b-b589-f225840245a3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:50.954140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 30/60] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:44:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:50.959673 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd1cc1-6886-48c0-b7c2-627be8a3a86b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:51.019950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:51.059413 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd1cc1-6886-48c0-b7c2-627be8a3a86b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:51.093043 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7a2a8-ba85-4013-964a-208b148f4f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:51.093713 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7a2a8-ba85-4013-964a-208b148f4f43 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:51.094006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7a2a8-ba85-4013-964a-208b148f4f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:51.094290 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7a2a8-ba85-4013-964a-208b148f4f43 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:51.095040 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7a2a8-ba85-4013-964a-208b148f4f43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:51.096502 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7a2a8-ba85-4013-964a-208b148f4f43 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:51.121865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 31/61] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:44:50 2022] POST /identity/v3/users => generated 275 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:44:54.436177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c62aa5-cb9d-415d-9286-bedb7b88b385 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:54.437185 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c62aa5-cb9d-415d-9286-bedb7b88b385 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:54.437678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c62aa5-cb9d-415d-9286-bedb7b88b385 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:54.439288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 31/62] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:44:54.440511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500d74d-be98-431d-af50-bc1a77683746 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:54.440888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500d74d-be98-431d-af50-bc1a77683746 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:54.441081 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500d74d-be98-431d-af50-bc1a77683746 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:54.442701 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 32/63] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:44:54.447741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f678d12-96e2-4204-b0c8-e7c712331731 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:54.448263 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f678d12-96e2-4204-b0c8-e7c712331731 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:54.449443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f678d12-96e2-4204-b0c8-e7c712331731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:54.452599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0eb788-6831-4780-9622-1752346fb9b6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:54.453127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0eb788-6831-4780-9622-1752346fb9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:54.453602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0eb788-6831-4780-9622-1752346fb9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:54.473062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2f678d12-96e2-4204-b0c8-e7c712331731 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:54.473600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ff0eb788-6831-4780-9622-1752346fb9b6 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:54.474790 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2f678d12-96e2-4204-b0c8-e7c712331731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:54.475272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ff0eb788-6831-4780-9622-1752346fb9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:54.550131 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 33/64] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:44:54.576201 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0eb788-6831-4780-9622-1752346fb9b6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:54.578004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:54.601231 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 32/65] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:44:54.646995 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:54.649048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f678d12-96e2-4204-b0c8-e7c712331731 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:54.706161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0eb788-6831-4780-9622-1752346fb9b6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:54.748396 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6a330-fc2f-4da0-935c-1cd8900ad61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:54.749691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6a330-fc2f-4da0-935c-1cd8900ad61e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:54.750297 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6a330-fc2f-4da0-935c-1cd8900ad61e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:54.750798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6a330-fc2f-4da0-935c-1cd8900ad61e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:54.763212 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6a330-fc2f-4da0-935c-1cd8900ad61e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:44:54.772615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-79b6a330-fc2f-4da0-935c-1cd8900ad61e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:44:54.772615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:44:54.772615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:44:54.772615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:44:54.772615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:44:54.772615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:44:54.772615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:44:54.772615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:44:54.772615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:44:54.772615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:44:54.772615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:44:54.772615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:44:54.772615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:44:54.772615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:44:54.772615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:44:54.772615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:44:54.773608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:44:54.773608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:44:54.773608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:44:54.773608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:44:54.773608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:44:54.773608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:44:54.773608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:44:54.773608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:44:54.773608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:44:54.773608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:44:54.773608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:44:54.773608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:44:54.773608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:44:54.773608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:44:54.773608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:44:54.773608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:44:54.774543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:44:54.774543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:44:54.774543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:44:54.774543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:44:54.774543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:44:54.774543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:44:54.774543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:44:54.774543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:44:54.774543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:44:54.774543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:44:54.774543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:44:54.774543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:44:54.779391 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 34/66] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:44:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:44:54.784180 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218171d-0b37-4446-982e-13717da8a437 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:54.784180 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218171d-0b37-4446-982e-13717da8a437 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:54.784180 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218171d-0b37-4446-982e-13717da8a437 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:54.786724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 35/67] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:44:54.788173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218171d-0b37-4446-982e-13717da8a437 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:54.835320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f678d12-96e2-4204-b0c8-e7c712331731 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:54.873559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b205296-fcd0-4bb5-a610-4e921b9c99fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:54.874775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b205296-fcd0-4bb5-a610-4e921b9c99fe None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:54.875417 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b205296-fcd0-4bb5-a610-4e921b9c99fe None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:54.875701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b205296-fcd0-4bb5-a610-4e921b9c99fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:54.888492 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b205296-fcd0-4bb5-a610-4e921b9c99fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:44:54.897216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-2b205296-fcd0-4bb5-a610-4e921b9c99fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:44:54.897216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:44:54.897216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:44:54.897216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:44:54.897216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:44:54.897216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:44:54.897216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:44:54.897216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:44:54.897216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:44:54.897216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:44:54.897216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:44:54.897216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:44:54.897216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:44:54.897216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:44:54.897216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:44:54.897216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:44:54.898075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:44:54.898075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:44:54.898075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:44:54.898075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:44:54.898075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:44:54.898075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:44:54.898075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:44:54.898075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:44:54.898075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:44:54.898075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:44:54.898075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:44:54.898075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:44:54.898075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:44:54.898075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:44:54.898075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:44:54.898075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:44:54.898873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:44:54.898873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:44:54.898873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:44:54.898873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:44:54.898873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:44:54.898873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:44:54.898873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:44:54.898873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:44:54.898873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:44:54.898873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:44:54.898873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:44:54.898873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:44:54.898873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 33/68] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:44:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:44:54.900373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68df98-50dd-4c4c-95e8-25714fba854b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:54.900373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68df98-50dd-4c4c-95e8-25714fba854b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:54.900373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68df98-50dd-4c4c-95e8-25714fba854b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:54.918271 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5b68df98-50dd-4c4c-95e8-25714fba854b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:54.919518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5b68df98-50dd-4c4c-95e8-25714fba854b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:54.934159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218171d-0b37-4446-982e-13717da8a437 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:54.944110 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:54.977977 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:54.981632 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254bae9-b34f-4e88-87f2-229c94e571cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:54.982268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254bae9-b34f-4e88-87f2-229c94e571cb None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:54.982560 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254bae9-b34f-4e88-87f2-229c94e571cb None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:54.982870 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254bae9-b34f-4e88-87f2-229c94e571cb None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:54.983624 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254bae9-b34f-4e88-87f2-229c94e571cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:44:54.984119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254bae9-b34f-4e88-87f2-229c94e571cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:54.985531 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254bae9-b34f-4e88-87f2-229c94e571cb None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:54.994704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 36/69] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:44:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:54.996677 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07812293-43a1-461d-96da-f9717a491f11 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:54.997045 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07812293-43a1-461d-96da-f9717a491f11 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:54.997401 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07812293-43a1-461d-96da-f9717a491f11 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:54.998762 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 37/70] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:44:55.003673 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07812293-43a1-461d-96da-f9717a491f11 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:55.006709 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 34/71] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:44:55.011829 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68df98-50dd-4c4c-95e8-25714fba854b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:55.117935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07812293-43a1-461d-96da-f9717a491f11 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:55.122068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68df98-50dd-4c4c-95e8-25714fba854b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:55.157238 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcdd56-c5ec-4bd6-bc1e-97e190c512f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:55.158624 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcdd56-c5ec-4bd6-bc1e-97e190c512f2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:55.159210 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcdd56-c5ec-4bd6-bc1e-97e190c512f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:55.159723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcdd56-c5ec-4bd6-bc1e-97e190c512f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:55.160894 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bcdd56-c5ec-4bd6-bc1e-97e190c512f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:44:55.161459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bcdd56-c5ec-4bd6-bc1e-97e190c512f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:55.161848 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d50c4b-28d6-4149-8695-1d87c1a718fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:55.162770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d50c4b-28d6-4149-8695-1d87c1a718fb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:55.163058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d50c4b-28d6-4149-8695-1d87c1a718fb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:55.163152 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bcdd56-c5ec-4bd6-bc1e-97e190c512f2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:55.163375 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d50c4b-28d6-4149-8695-1d87c1a718fb None admin] PATH_INFO: `/v3/users/7e2576df31cd4791832f5441b6f98ba5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:55.164781 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d50c4b-28d6-4149-8695-1d87c1a718fb None admin] RBAC: Authorizing `identity:get_user(user_id=7e2576df31cd4791832f5441b6f98ba5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:55.166101 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d50c4b-28d6-4149-8695-1d87c1a718fb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:55.168986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 35/72] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:44:55 2022] GET /identity/v3/users/7e2576df31cd4791832f5441b6f98ba5 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:55.171969 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac2ee5-9dcb-4503-9d8a-319432d803f2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:55.172475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac2ee5-9dcb-4503-9d8a-319432d803f2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:55.172978 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac2ee5-9dcb-4503-9d8a-319432d803f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:55.173390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 38/73] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:44:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:55.174209 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcdd56-c5ec-4bd6-bc1e-97e190c512f2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:55.192616 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d5ac2ee5-9dcb-4503-9d8a-319432d803f2 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:55.193916 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d5ac2ee5-9dcb-4503-9d8a-319432d803f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:55.208376 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:55.243995 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:55.270198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 36/74] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:44:55.271385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac2ee5-9dcb-4503-9d8a-319432d803f2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:55.292376 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcdd56-c5ec-4bd6-bc1e-97e190c512f2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:55.333145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617acf2-cddb-4676-b239-c8b9ecf63e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:55.334087 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617acf2-cddb-4676-b239-c8b9ecf63e53 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:55.334379 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617acf2-cddb-4676-b239-c8b9ecf63e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:55.334689 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617acf2-cddb-4676-b239-c8b9ecf63e53 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:55.345523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617acf2-cddb-4676-b239-c8b9ecf63e53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:44:55.353556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-f617acf2-cddb-4676-b239-c8b9ecf63e53 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:44:55.353556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:44:55.353556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:44:55.353556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:44:55.353556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:44:55.353556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:44:55.353556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:44:55.353556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:44:55.353556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:44:55.353556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:44:55.353556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:44:55.353556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:44:55.353556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:44:55.353556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:44:55.353556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:44:55.353556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:44:55.354395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:44:55.354395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:44:55.354395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:44:55.354395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:44:55.354395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:44:55.354395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:44:55.354395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:44:55.354395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:44:55.354395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:44:55.354395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:44:55.354395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:44:55.354395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:44:55.354395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:44:55.354395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:44:55.354395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:44:55.354395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:44:55.355690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:44:55.355690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:44:55.355690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:44:55.355690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:44:55.355690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:44:55.355690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:44:55.355690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:44:55.355690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:44:55.355690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:44:55.355690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:44:55.355690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:44:55.355690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:44:55.355690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 39/75] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:44:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:44:55.356584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617acf2-cddb-4676-b239-c8b9ecf63e53 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:55.391671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac2ee5-9dcb-4503-9d8a-319432d803f2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:55.426877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d8cce-fd5d-4ed9-834c-1b730a024ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:55.427539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d8cce-fd5d-4ed9-834c-1b730a024ebd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:55.427833 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d8cce-fd5d-4ed9-834c-1b730a024ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:55.428115 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d8cce-fd5d-4ed9-834c-1b730a024ebd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:55.448349 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:55.457854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617acf2-cddb-4676-b239-c8b9ecf63e53 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:55.474225 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:55.492328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236b0c3-e35d-40a0-bcaf-a52e29d7e07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:55.493202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236b0c3-e35d-40a0-bcaf-a52e29d7e07f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:55.493494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236b0c3-e35d-40a0-bcaf-a52e29d7e07f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:55.493774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236b0c3-e35d-40a0-bcaf-a52e29d7e07f None admin] PATH_INFO: `/v3/users/c02531851c6c49c9abab77229f96d67f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:55.497363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d8cce-fd5d-4ed9-834c-1b730a024ebd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:55.498824 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d8cce-fd5d-4ed9-834c-1b730a024ebd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:55.507772 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236b0c3-e35d-40a0-bcaf-a52e29d7e07f None admin] RBAC: Authorizing `identity:get_user(user_id=c02531851c6c49c9abab77229f96d67f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:55.509248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236b0c3-e35d-40a0-bcaf-a52e29d7e07f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:55.510967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 40/76] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:44:55 2022] GET /identity/v3/users/c02531851c6c49c9abab77229f96d67f => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:55.512475 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236b0c3-e35d-40a0-bcaf-a52e29d7e07f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:55.599034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 37/77] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:44:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:44:55.600569 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9978e-c3bb-4211-900f-12e4cdc9b7a9 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:55.600870 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9978e-c3bb-4211-900f-12e4cdc9b7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:55.601156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9978e-c3bb-4211-900f-12e4cdc9b7a9 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:55.602449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 38/78] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:44:55.603182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9978e-c3bb-4211-900f-12e4cdc9b7a9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:55.619392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236b0c3-e35d-40a0-bcaf-a52e29d7e07f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:55.655058 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa28213-ea3d-4934-98b9-b2515d928ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:55.655860 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa28213-ea3d-4934-98b9-b2515d928ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:55.656148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa28213-ea3d-4934-98b9-b2515d928ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:55.656429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa28213-ea3d-4934-98b9-b2515d928ab0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:55.666498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa28213-ea3d-4934-98b9-b2515d928ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:44:55.675276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-dfa28213-ea3d-4934-98b9-b2515d928ab0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:44:55.675276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:44:55.675276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:44:55.675276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:44:55.675276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:44:55.675276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:44:55.675276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:44:55.675276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:44:55.675276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:44:55.675276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:44:55.675276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:44:55.675276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:44:55.675276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:44:55.675276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:44:55.675276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:44:55.675276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:44:55.676123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:44:55.676123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:44:55.676123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:44:55.676123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:44:55.676123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:44:55.676123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:44:55.676123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:44:55.676123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:44:55.676123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:44:55.676123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:44:55.676123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:44:55.676123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:44:55.676123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:44:55.676123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:44:55.676123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:44:55.676123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:44:55.677127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:44:55.677127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:44:55.677127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:44:55.677127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:44:55.677127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:44:55.677127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:44:55.677127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:44:55.677127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:44:55.677127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:44:55.677127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:44:55.677127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:44:55.677127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:44:55.677127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 41/79] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:44:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:44:55.679416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa28213-ea3d-4934-98b9-b2515d928ab0 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:55.686054 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:55.692157 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:55.709689 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9978e-c3bb-4211-900f-12e4cdc9b7a9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:55.743967 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d8013-9b39-4939-ab44-f4048ccc201d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:55.744640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d8013-9b39-4939-ab44-f4048ccc201d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:55.744925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d8013-9b39-4939-ab44-f4048ccc201d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:55.745205 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d8013-9b39-4939-ab44-f4048ccc201d None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:55.746537 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d8013-9b39-4939-ab44-f4048ccc201d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:44:55.746871 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d8013-9b39-4939-ab44-f4048ccc201d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:55.748322 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d8013-9b39-4939-ab44-f4048ccc201d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:55.759690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 39/80] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:44:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:55.760682 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d8013-9b39-4939-ab44-f4048ccc201d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:55.781149 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa28213-ea3d-4934-98b9-b2515d928ab0 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:55.815482 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb35b9-4db7-4d1e-b97d-1def080ed2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:55.816142 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb35b9-4db7-4d1e-b97d-1def080ed2fe None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:55.816430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb35b9-4db7-4d1e-b97d-1def080ed2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:55.816710 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb35b9-4db7-4d1e-b97d-1def080ed2fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:55.864967 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d8013-9b39-4939-ab44-f4048ccc201d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:55.887669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb35b9-4db7-4d1e-b97d-1def080ed2fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:55.889148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb35b9-4db7-4d1e-b97d-1def080ed2fe None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:55.900240 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:55.902562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df7e89-c204-43bc-8a7d-e8fab7a03194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:55.903467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df7e89-c204-43bc-8a7d-e8fab7a03194 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:55.903759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df7e89-c204-43bc-8a7d-e8fab7a03194 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:55.904042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df7e89-c204-43bc-8a7d-e8fab7a03194 None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:55.905274 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df7e89-c204-43bc-8a7d-e8fab7a03194 None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:55.906757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df7e89-c204-43bc-8a7d-e8fab7a03194 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:55.908469 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 40/81] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:44:55 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:55.909614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123f550-92a9-425f-b9f2-d3d2e4a65e67 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:55.909903 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123f550-92a9-425f-b9f2-d3d2e4a65e67 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:55.910184 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123f550-92a9-425f-b9f2-d3d2e4a65e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:55.928531 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5123f550-92a9-425f-b9f2-d3d2e4a65e67 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:55.929746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5123f550-92a9-425f-b9f2-d3d2e4a65e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:55.931753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:55.993440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 42/82] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:44:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:44:55.994288 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb35b9-4db7-4d1e-b97d-1def080ed2fe None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:56.000113 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 41/83] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:44:56.001092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123f550-92a9-425f-b9f2-d3d2e4a65e67 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:56.099717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123f550-92a9-425f-b9f2-d3d2e4a65e67 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:56.101892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb35b9-4db7-4d1e-b97d-1def080ed2fe None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:56.113833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:56.134416 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb5018-754d-4c54-81dd-342afb366ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:56.135318 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb5018-754d-4c54-81dd-342afb366ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:56.135601 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb5018-754d-4c54-81dd-342afb366ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:56.135874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb5018-754d-4c54-81dd-342afb366ce1 None admin] PATH_INFO: `/v3/users/c02531851c6c49c9abab77229f96d67f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:56.137141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb5018-754d-4c54-81dd-342afb366ce1 None admin] RBAC: Authorizing `identity:get_user(user_id=c02531851c6c49c9abab77229f96d67f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:56.138607 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb5018-754d-4c54-81dd-342afb366ce1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:56.138773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67f640-0403-44ea-adf5-63655e66b504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:56.139439 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67f640-0403-44ea-adf5-63655e66b504 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:56.139731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67f640-0403-44ea-adf5-63655e66b504 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:56.140008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67f640-0403-44ea-adf5-63655e66b504 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:56.140223 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 42/84] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:44:56 2022] GET /identity/v3/users/c02531851c6c49c9abab77229f96d67f => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:56.140752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67f640-0403-44ea-adf5-63655e66b504 None admin] RBAC: Adding query filter params (name=member) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:44:56.140979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb5018-754d-4c54-81dd-342afb366ce1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:56.141072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67f640-0403-44ea-adf5-63655e66b504 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:56.142423 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67f640-0403-44ea-adf5-63655e66b504 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:56.149594 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:56.152699 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 43/85] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:44:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:56.153522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67f640-0403-44ea-adf5-63655e66b504 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:56.243475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb5018-754d-4c54-81dd-342afb366ce1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:56.253699 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67f640-0403-44ea-adf5-63655e66b504 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:56.277826 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9247a4-29e1-4555-8f35-68491aafec9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:56.278486 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9247a4-29e1-4555-8f35-68491aafec9c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:56.278789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9247a4-29e1-4555-8f35-68491aafec9c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:56.279063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9247a4-29e1-4555-8f35-68491aafec9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:56.279885 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9247a4-29e1-4555-8f35-68491aafec9c None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=aa7a11e47f684403b0ebf54d869da5be, user.id=7e2576df31cd4791832f5441b6f98ba5) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:44:56.280197 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9247a4-29e1-4555-8f35-68491aafec9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:56.281628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9247a4-29e1-4555-8f35-68491aafec9c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:56.288035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0789673-7eb2-4cd5-bcff-a7a0b5152cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:56.288948 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0789673-7eb2-4cd5-bcff-a7a0b5152cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:56.289106 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 43/86] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:44:56 2022] GET /identity/v3/role_assignments?user.id=7e2576df31cd4791832f5441b6f98ba5&scope.project.id=aa7a11e47f684403b0ebf54d869da5be&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:56.289880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9247a4-29e1-4555-8f35-68491aafec9c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:56.290691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0789673-7eb2-4cd5-bcff-a7a0b5152cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:56.291018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0789673-7eb2-4cd5-bcff-a7a0b5152cf3 None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:56.292695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0789673-7eb2-4cd5-bcff-a7a0b5152cf3 None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:56.295144 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0789673-7eb2-4cd5-bcff-a7a0b5152cf3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:56.297228 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 44/87] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:44:56 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:56.297825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0789673-7eb2-4cd5-bcff-a7a0b5152cf3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:56.349372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:56.375181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:56.411651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9247a4-29e1-4555-8f35-68491aafec9c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:56.417268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0789673-7eb2-4cd5-bcff-a7a0b5152cf3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:56.452563 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae99110-697b-44c3-b0e2-4258648a68de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:56.454116 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae99110-697b-44c3-b0e2-4258648a68de None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:56.454554 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae99110-697b-44c3-b0e2-4258648a68de None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:56.455079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae99110-697b-44c3-b0e2-4258648a68de None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:56.456426 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae99110-697b-44c3-b0e2-4258648a68de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:56.458106 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abd774-3e9a-433b-b768-5a3014f145df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:56.458632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae99110-697b-44c3-b0e2-4258648a68de None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:56.458823 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abd774-3e9a-433b-b768-5a3014f145df None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:56.459155 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abd774-3e9a-433b-b768-5a3014f145df None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:56.459442 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abd774-3e9a-433b-b768-5a3014f145df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:56.460294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 44/88] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:44:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:56.461060 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae99110-697b-44c3-b0e2-4258648a68de None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:56.534647 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abd774-3e9a-433b-b768-5a3014f145df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:56.536057 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abd774-3e9a-433b-b768-5a3014f145df None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:56.572403 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae99110-697b-44c3-b0e2-4258648a68de None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:56.579174 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:56.611177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a678e83-be5a-45eb-a172-1a8d499d6c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:56.612056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a678e83-be5a-45eb-a172-1a8d499d6c88 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:56.612345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a678e83-be5a-45eb-a172-1a8d499d6c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:56.612623 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a678e83-be5a-45eb-a172-1a8d499d6c88 None admin] PATH_INFO: `/v3/users/c02531851c6c49c9abab77229f96d67f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:56.613830 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a678e83-be5a-45eb-a172-1a8d499d6c88 None admin] RBAC: Authorizing `identity:get_user(user_id=c02531851c6c49c9abab77229f96d67f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:56.615731 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a678e83-be5a-45eb-a172-1a8d499d6c88 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:56.617765 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 45/89] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:44:56 2022] GET /identity/v3/users/c02531851c6c49c9abab77229f96d67f => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:56.619378 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a678e83-be5a-45eb-a172-1a8d499d6c88 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:56.629916 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:56.647725 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 45/90] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:44:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:44:56.653455 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abd774-3e9a-433b-b768-5a3014f145df None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:56.723486 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a678e83-be5a-45eb-a172-1a8d499d6c88 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:56.760264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a575591-e718-48fb-a34e-0eac0a6a51cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:56.762087 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a575591-e718-48fb-a34e-0eac0a6a51cb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:56.762683 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a575591-e718-48fb-a34e-0eac0a6a51cb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:56.763191 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a575591-e718-48fb-a34e-0eac0a6a51cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:56.764212 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a575591-e718-48fb-a34e-0eac0a6a51cb None admin] RBAC: Adding query filter params (role.id=cf55e7c6398f45efaf10dec4fe153f8a, scope.project.id=aa7a11e47f684403b0ebf54d869da5be, user.id=c02531851c6c49c9abab77229f96d67f) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:44:56.764741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a575591-e718-48fb-a34e-0eac0a6a51cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:56.766282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abd774-3e9a-433b-b768-5a3014f145df None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:56.766617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a575591-e718-48fb-a34e-0eac0a6a51cb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:56.773437 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 46/91] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:44:56 2022] GET /identity/v3/role_assignments?user.id=c02531851c6c49c9abab77229f96d67f&scope.project.id=aa7a11e47f684403b0ebf54d869da5be&role.id=cf55e7c6398f45efaf10dec4fe153f8a => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:56.774453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a575591-e718-48fb-a34e-0eac0a6a51cb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:56.821567 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3845296-f5d2-4f3e-accd-edfad4b280c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:56.822184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3845296-f5d2-4f3e-accd-edfad4b280c7 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:56.822476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3845296-f5d2-4f3e-accd-edfad4b280c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:56.822792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3845296-f5d2-4f3e-accd-edfad4b280c7 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:56.824551 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3845296-f5d2-4f3e-accd-edfad4b280c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:56.826618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3845296-f5d2-4f3e-accd-edfad4b280c7 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:56.838472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:56.854396 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 46/92] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:44:56 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:44:56.855523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3845296-f5d2-4f3e-accd-edfad4b280c7 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:56.948618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:57.001439 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3845296-f5d2-4f3e-accd-edfad4b280c7 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:57.024940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a575591-e718-48fb-a34e-0eac0a6a51cb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:57.064035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e67a95-3015-441a-983a-e6f7a7e1617d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:57.065080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e67a95-3015-441a-983a-e6f7a7e1617d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:57.065352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e67a95-3015-441a-983a-e6f7a7e1617d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:57.065634 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e67a95-3015-441a-983a-e6f7a7e1617d None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:57.068554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e67a95-3015-441a-983a-e6f7a7e1617d None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:57.070472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e67a95-3015-441a-983a-e6f7a7e1617d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:57.073643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 47/93] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:44:56 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:57.080743 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e67a95-3015-441a-983a-e6f7a7e1617d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:57.092533 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03888338-927e-4505-9fae-3bc3956b8002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:57.093204 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03888338-927e-4505-9fae-3bc3956b8002 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:57.093493 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03888338-927e-4505-9fae-3bc3956b8002 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:57.093774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03888338-927e-4505-9fae-3bc3956b8002 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:57.195163 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:57.207659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03888338-927e-4505-9fae-3bc3956b8002 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:57.209460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03888338-927e-4505-9fae-3bc3956b8002 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:57.254249 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e67a95-3015-441a-983a-e6f7a7e1617d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:57.302021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3eab4-0e1d-4b5e-ab5e-b4c9b918b3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:57.302737 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3eab4-0e1d-4b5e-ab5e-b4c9b918b3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:57.303027 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3eab4-0e1d-4b5e-ab5e-b4c9b918b3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:57.303373 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3eab4-0e1d-4b5e-ab5e-b4c9b918b3b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:57.304248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3eab4-0e1d-4b5e-ab5e-b4c9b918b3b1 None admin] RBAC: Adding query filter params (role.id=662cb9b5dff94cdb83f292ad20d1d430, scope.project.id=aa7a11e47f684403b0ebf54d869da5be, user.id=c02531851c6c49c9abab77229f96d67f) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:44:57.304580 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3eab4-0e1d-4b5e-ab5e-b4c9b918b3b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:57.306257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3eab4-0e1d-4b5e-ab5e-b4c9b918b3b1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:57.310834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:57.315807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 48/94] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:44:57 2022] GET /identity/v3/role_assignments?user.id=c02531851c6c49c9abab77229f96d67f&scope.project.id=aa7a11e47f684403b0ebf54d869da5be&role.id=662cb9b5dff94cdb83f292ad20d1d430 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:44:57.361424 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 47/95] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:44:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:44:57.367395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3eab4-0e1d-4b5e-ab5e-b4c9b918b3b1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:57.497241 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3eab4-0e1d-4b5e-ab5e-b4c9b918b3b1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:57.537425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:57.550181 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071fada-1198-4ec7-a5f6-89dd03d011e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:57.551385 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071fada-1198-4ec7-a5f6-89dd03d011e6 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:57.551844 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071fada-1198-4ec7-a5f6-89dd03d011e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:57.552272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071fada-1198-4ec7-a5f6-89dd03d011e6 None admin] PATH_INFO: `/v3/projects/42c0dc6e486e4990b8c551043685c92d` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:57.553697 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071fada-1198-4ec7-a5f6-89dd03d011e6 None admin] RBAC: Authorizing `identity:get_project(project_id=42c0dc6e486e4990b8c551043685c92d)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:57.555354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071fada-1198-4ec7-a5f6-89dd03d011e6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:57.557269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 49/96] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:44:57 2022] GET /identity/v3/projects/42c0dc6e486e4990b8c551043685c92d => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:44:57.563541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03888338-927e-4505-9fae-3bc3956b8002 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:57.682012 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03888338-927e-4505-9fae-3bc3956b8002 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:57.725541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a45a2-457f-44e2-8203-f938d24265e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:57.726451 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a45a2-457f-44e2-8203-f938d24265e6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:57.726908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a45a2-457f-44e2-8203-f938d24265e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:57.727360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a45a2-457f-44e2-8203-f938d24265e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:57.728344 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a45a2-457f-44e2-8203-f938d24265e6 None admin] RBAC: Adding query filter params (role.id=cf55e7c6398f45efaf10dec4fe153f8a, scope.project.id=42c0dc6e486e4990b8c551043685c92d, user.id=c02531851c6c49c9abab77229f96d67f) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:44:57.728804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a45a2-457f-44e2-8203-f938d24265e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:57.730455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a45a2-457f-44e2-8203-f938d24265e6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:57.742917 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 48/97] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:44:57 2022] GET /identity/v3/role_assignments?user.id=c02531851c6c49c9abab77229f96d67f&scope.project.id=42c0dc6e486e4990b8c551043685c92d&role.id=cf55e7c6398f45efaf10dec4fe153f8a => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:44:59.069200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b9638-3b73-4b43-aa87-0f07fb9f4e05 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:59.070306 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b9638-3b73-4b43-aa87-0f07fb9f4e05 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:59.070837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b9638-3b73-4b43-aa87-0f07fb9f4e05 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:59.072430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 50/98] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:44:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:44:59.079212 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d053264-1bba-4316-ba3c-f45495ee7eff None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:59.080103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d053264-1bba-4316-ba3c-f45495ee7eff None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:59.080594 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d053264-1bba-4316-ba3c-f45495ee7eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:59.102268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1d053264-1bba-4316-ba3c-f45495ee7eff None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:59.103843 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1d053264-1bba-4316-ba3c-f45495ee7eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:59.139640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:59.191299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 49/99] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:44:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:44:59.226606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b9638-3b73-4b43-aa87-0f07fb9f4e05 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:59.360285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b9638-3b73-4b43-aa87-0f07fb9f4e05 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:59.401813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a436289-3891-43a7-b929-94098df6aa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:59.403263 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a436289-3891-43a7-b929-94098df6aa3f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:59.403754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a436289-3891-43a7-b929-94098df6aa3f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:59.404268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a436289-3891-43a7-b929-94098df6aa3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:59.422858 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a436289-3891-43a7-b929-94098df6aa3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:44:59.430837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-3a436289-3891-43a7-b929-94098df6aa3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:44:59.430837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:44:59.430837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:44:59.430837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:44:59.430837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:44:59.430837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:44:59.430837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:44:59.430837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:44:59.430837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:44:59.430837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:44:59.430837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:44:59.430837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:44:59.430837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:44:59.430837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:44:59.430837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:44:59.430837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:44:59.431731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:44:59.431731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:44:59.431731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:44:59.431731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:44:59.431731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:44:59.431731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:44:59.431731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:44:59.431731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:44:59.431731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:44:59.431731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:44:59.431731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:44:59.431731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:44:59.431731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:44:59.431731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:44:59.431731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:44:59.431731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:44:59.432515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:44:59.432515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:44:59.432515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:44:59.432515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:44:59.432515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:44:59.432515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:44:59.432515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:44:59.432515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:44:59.432515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:44:59.432515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:44:59.432515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:44:59.432515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:44:59.435583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 51/100] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:44:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:44:59.443690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d053264-1bba-4316-ba3c-f45495ee7eff None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:59.590588 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d053264-1bba-4316-ba3c-f45495ee7eff None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:59.635361 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b7a79-a761-466c-ba6c-b266abc7ce1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:59.637118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b7a79-a761-466c-ba6c-b266abc7ce1c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:59.637609 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b7a79-a761-466c-ba6c-b266abc7ce1c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:59.638080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b7a79-a761-466c-ba6c-b266abc7ce1c None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:59.639069 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b7a79-a761-466c-ba6c-b266abc7ce1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:44:59.639595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b7a79-a761-466c-ba6c-b266abc7ce1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:59.641314 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b7a79-a761-466c-ba6c-b266abc7ce1c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:59.654781 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 50/101] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:44:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:59.662192 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a436289-3891-43a7-b929-94098df6aa3f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:59.738035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:59.791573 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a436289-3891-43a7-b929-94098df6aa3f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:59.836255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3958b19-a437-4485-a3f0-04bd7cdb5733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:59.837213 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3958b19-a437-4485-a3f0-04bd7cdb5733 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:59.837502 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3958b19-a437-4485-a3f0-04bd7cdb5733 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:59.837781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3958b19-a437-4485-a3f0-04bd7cdb5733 None admin] PATH_INFO: `/v3/users/7e2576df31cd4791832f5441b6f98ba5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:59.839119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3958b19-a437-4485-a3f0-04bd7cdb5733 None admin] RBAC: Authorizing `identity:get_user(user_id=7e2576df31cd4791832f5441b6f98ba5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:59.840738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3958b19-a437-4485-a3f0-04bd7cdb5733 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:59.842496 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 52/102] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:44:59 2022] GET /identity/v3/users/7e2576df31cd4791832f5441b6f98ba5 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:44:59.848372 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b7a79-a761-466c-ba6c-b266abc7ce1c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:59.891857 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:00.000191 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b7a79-a761-466c-ba6c-b266abc7ce1c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:00.047701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74372d9-1fc2-42f9-a98c-1e7a9315f8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:00.048588 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74372d9-1fc2-42f9-a98c-1e7a9315f8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:00.048870 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74372d9-1fc2-42f9-a98c-1e7a9315f8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:00.049147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74372d9-1fc2-42f9-a98c-1e7a9315f8d9 None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:00.056033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f84e83-c2ab-4b59-89d1-4b2b3635eb3b None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:00.056341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f84e83-c2ab-4b59-89d1-4b2b3635eb3b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:00.056598 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f84e83-c2ab-4b59-89d1-4b2b3635eb3b None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:00.058027 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 53/103] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:45:00.059200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74372d9-1fc2-42f9-a98c-1e7a9315f8d9 None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:00.060687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74372d9-1fc2-42f9-a98c-1e7a9315f8d9 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:00.062840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 51/104] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:44:59 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:00.068327 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f84e83-c2ab-4b59-89d1-4b2b3635eb3b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:00.078428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c0104-48f7-438a-80bd-6709ad939c43 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:00.078743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c0104-48f7-438a-80bd-6709ad939c43 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:00.079022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c0104-48f7-438a-80bd-6709ad939c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:00.124500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c27c0104-48f7-438a-80bd-6709ad939c43 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:00.125616 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c27c0104-48f7-438a-80bd-6709ad939c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:00.301603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 52/105] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:00.309595 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f84e83-c2ab-4b59-89d1-4b2b3635eb3b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:00.337916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:00.346359 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c0104-48f7-438a-80bd-6709ad939c43 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:00.365283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4b516-5fbc-4ad1-8530-4edc8c7837ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:00.366561 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4b516-5fbc-4ad1-8530-4edc8c7837ea None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:00.367192 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4b516-5fbc-4ad1-8530-4edc8c7837ea None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:00.367795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4b516-5fbc-4ad1-8530-4edc8c7837ea None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be/users/7e2576df31cd4791832f5441b6f98ba5/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:00.376567 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4b516-5fbc-4ad1-8530-4edc8c7837ea None admin] RBAC: Authorizing `identity:create_grant(project_id=aa7a11e47f684403b0ebf54d869da5be, user_id=7e2576df31cd4791832f5441b6f98ba5, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:00.377965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4b516-5fbc-4ad1-8530-4edc8c7837ea None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:00.392921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:00.404710 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 54/106] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:45:00 2022] PUT /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be/users/7e2576df31cd4791832f5441b6f98ba5/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:45:00.407375 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544dc33-da89-49d6-88a9-bbab47640bbb None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:00.408454 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544dc33-da89-49d6-88a9-bbab47640bbb None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:00.408743 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544dc33-da89-49d6-88a9-bbab47640bbb None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:00.413707 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 55/107] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:00.420221 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1521b44-7d40-4280-acbe-0b01e854a23b None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:00.421365 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1521b44-7d40-4280-acbe-0b01e854a23b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:00.421662 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1521b44-7d40-4280-acbe-0b01e854a23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:00.454116 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e1521b44-7d40-4280-acbe-0b01e854a23b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:00.459670 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e1521b44-7d40-4280-acbe-0b01e854a23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:00.557321 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c0104-48f7-438a-80bd-6709ad939c43 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:00.602047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 56/108] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:00.613990 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6911be9-2e0f-4c54-9950-4abf03374b49 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:00.614343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6911be9-2e0f-4c54-9950-4abf03374b49 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:00.614648 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6911be9-2e0f-4c54-9950-4abf03374b49 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:00.616779 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 57/109] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:45:00.623272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6ee4b-3fa2-46f5-99cf-cd69a54575a9 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:00.623708 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6ee4b-3fa2-46f5-99cf-cd69a54575a9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:00.624134 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6ee4b-3fa2-46f5-99cf-cd69a54575a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:00.647302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59f4bc-6839-444f-a53c-462d6997037e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:00.648748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59f4bc-6839-444f-a53c-462d6997037e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:00.649042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59f4bc-6839-444f-a53c-462d6997037e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:00.649404 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59f4bc-6839-444f-a53c-462d6997037e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:00.671548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-37a6ee4b-3fa2-46f5-99cf-cd69a54575a9 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:00.672798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-37a6ee4b-3fa2-46f5-99cf-cd69a54575a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:00.676448 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59f4bc-6839-444f-a53c-462d6997037e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:00.685315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-6e59f4bc-6839-444f-a53c-462d6997037e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:00.685315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:00.685315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:00.685315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:00.685315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:00.685315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:00.685315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:00.685315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:00.685315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:00.685315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:00.685315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:00.685315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:00.685315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:00.685315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:00.685315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:00.685315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:00.686282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:00.686282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:00.686282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:00.686282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:00.686282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:00.686282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:00.686282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:00.686282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:00.686282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:00.686282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:00.686282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:00.686282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:00.686282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:00.686282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:00.686282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:00.686282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:00.687083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:00.687083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:00.687083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:00.687083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:00.687083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:00.687083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:00.687083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:00.687083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:00.687083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:00.687083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:00.687083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:00.687083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:00.696356 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 53/110] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:45:00.697385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59f4bc-6839-444f-a53c-462d6997037e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:00.783532 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 58/111] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:00.784232 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6ee4b-3fa2-46f5-99cf-cd69a54575a9 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:00.815415 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:00.832805 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59f4bc-6839-444f-a53c-462d6997037e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:00.858985 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:00.874602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538de2d-8c55-49dd-8e57-e810f83349c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:00.875542 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538de2d-8c55-49dd-8e57-e810f83349c8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:00.875829 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538de2d-8c55-49dd-8e57-e810f83349c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:00.876103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538de2d-8c55-49dd-8e57-e810f83349c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:00.877295 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3538de2d-8c55-49dd-8e57-e810f83349c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:00.879039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3538de2d-8c55-49dd-8e57-e810f83349c8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:00.880955 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 54/112] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:00.882078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df416b0-2e01-457d-af46-29c1b4be1e4c None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:00.886630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df416b0-2e01-457d-af46-29c1b4be1e4c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:00.886630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df416b0-2e01-457d-af46-29c1b4be1e4c None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:00.886630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 55/113] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:00.886630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df416b0-2e01-457d-af46-29c1b4be1e4c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:00.937972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6ee4b-3fa2-46f5-99cf-cd69a54575a9 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:00.981801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf36ec-4757-41f7-b5e8-483c2ff112f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:00.983408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf36ec-4757-41f7-b5e8-483c2ff112f1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:00.984169 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf36ec-4757-41f7-b5e8-483c2ff112f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:00.984648 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf36ec-4757-41f7-b5e8-483c2ff112f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:00.985631 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf36ec-4757-41f7-b5e8-483c2ff112f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:00.986135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf36ec-4757-41f7-b5e8-483c2ff112f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:00.988871 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf36ec-4757-41f7-b5e8-483c2ff112f1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:01.001950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 59/114] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:01.003593 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf36ec-4757-41f7-b5e8-483c2ff112f1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:01.034854 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df416b0-2e01-457d-af46-29c1b4be1e4c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:01.073577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e74a68-14f4-4d21-9c79-36acfaf41816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:01.075349 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e74a68-14f4-4d21-9c79-36acfaf41816 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:01.075839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e74a68-14f4-4d21-9c79-36acfaf41816 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:01.076328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e74a68-14f4-4d21-9c79-36acfaf41816 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:01.087810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e74a68-14f4-4d21-9c79-36acfaf41816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:01.097177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-23e74a68-14f4-4d21-9c79-36acfaf41816 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:01.097177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:01.097177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:01.097177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:01.097177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:01.097177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:01.097177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:01.097177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:01.097177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:01.097177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:01.097177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:01.097177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:01.097177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:01.097177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:01.097177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:01.097177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:01.098041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:01.098041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:01.098041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:01.098041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:01.098041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:01.098041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:01.098041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:01.098041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:01.098041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:01.098041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:01.098041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:01.098041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:01.098041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:01.098041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:01.098041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:01.098041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:01.099198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:01.099198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:01.099198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:01.099198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:01.099198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:01.099198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:01.099198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:01.099198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:01.099198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:01.099198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:01.099198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:01.099198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:01.101274 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 56/115] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:45:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:01.102716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff3fa7-b51a-48c4-b512-e6b5a368c357 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:01.103215 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff3fa7-b51a-48c4-b512-e6b5a368c357 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:01.103774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff3fa7-b51a-48c4-b512-e6b5a368c357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:01.125167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e6ff3fa7-b51a-48c4-b512-e6b5a368c357 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:01.127590 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e6ff3fa7-b51a-48c4-b512-e6b5a368c357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:01.136145 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:01.136344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf36ec-4757-41f7-b5e8-483c2ff112f1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:01.166808 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:01.177925 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e8da4-5883-4adc-9a43-082f2372d5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:01.178854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e8da4-5883-4adc-9a43-082f2372d5ae None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:01.179236 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e8da4-5883-4adc-9a43-082f2372d5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:01.179653 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e8da4-5883-4adc-9a43-082f2372d5ae None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:01.180690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e8da4-5883-4adc-9a43-082f2372d5ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:01.182269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e8da4-5883-4adc-9a43-082f2372d5ae None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:01.208950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 57/116] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:01.210010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff3fa7-b51a-48c4-b512-e6b5a368c357 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:01.216412 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 60/117] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Jun 16 18:45:01 2022] POST /identity/v3/projects => generated 302 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:45:01.226606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e8da4-5883-4adc-9a43-082f2372d5ae None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:01.350861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e8da4-5883-4adc-9a43-082f2372d5ae None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:01.376877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff3fa7-b51a-48c4-b512-e6b5a368c357 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:01.400017 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbac57-339e-4680-abc9-9561903eb9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:01.400858 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbac57-339e-4680-abc9-9561903eb9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:01.401239 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbac57-339e-4680-abc9-9561903eb9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:01.401731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbac57-339e-4680-abc9-9561903eb9f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:01.402648 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbac57-339e-4680-abc9-9561903eb9f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:01.403104 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbac57-339e-4680-abc9-9561903eb9f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:01.404583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbac57-339e-4680-abc9-9561903eb9f2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:01.421150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 61/118] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:45:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:01.421150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbac57-339e-4680-abc9-9561903eb9f2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:01.423726 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb85a81-81b9-403c-8dd0-e0ef418068c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:01.424869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb85a81-81b9-403c-8dd0-e0ef418068c5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:01.425354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb85a81-81b9-403c-8dd0-e0ef418068c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:01.425806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb85a81-81b9-403c-8dd0-e0ef418068c5 None admin] PATH_INFO: `/v3/users/c02531851c6c49c9abab77229f96d67f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:01.427317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb85a81-81b9-403c-8dd0-e0ef418068c5 None admin] RBAC: Authorizing `identity:get_user(user_id=c02531851c6c49c9abab77229f96d67f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:01.429014 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb85a81-81b9-403c-8dd0-e0ef418068c5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:01.431017 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 58/119] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:01 2022] GET /identity/v3/users/c02531851c6c49c9abab77229f96d67f => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:01.432515 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c716b25-b7b2-46b1-8bd4-01936ce5a52b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:01.433098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c716b25-b7b2-46b1-8bd4-01936ce5a52b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:01.433808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c716b25-b7b2-46b1-8bd4-01936ce5a52b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:01.435509 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 59/120] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:01.436410 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c716b25-b7b2-46b1-8bd4-01936ce5a52b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:01.451006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:01.454866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:01.561536 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c716b25-b7b2-46b1-8bd4-01936ce5a52b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:01.562992 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbac57-339e-4680-abc9-9561903eb9f2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:01.607979 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b709da-ccaa-428e-99be-2d441f18bf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:01.608780 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8300616-da3c-4303-b713-8e940bdb3f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:01.608901 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b709da-ccaa-428e-99be-2d441f18bf11 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:01.609083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b709da-ccaa-428e-99be-2d441f18bf11 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:01.609340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b709da-ccaa-428e-99be-2d441f18bf11 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:01.609670 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8300616-da3c-4303-b713-8e940bdb3f8d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:01.610142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8300616-da3c-4303-b713-8e940bdb3f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:01.610423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8300616-da3c-4303-b713-8e940bdb3f8d None admin] PATH_INFO: `/v3/users/c02531851c6c49c9abab77229f96d67f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:01.611932 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8300616-da3c-4303-b713-8e940bdb3f8d None admin] RBAC: Authorizing `identity:get_user(user_id=c02531851c6c49c9abab77229f96d67f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:01.613761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8300616-da3c-4303-b713-8e940bdb3f8d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:01.615955 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 60/121] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:01 2022] GET /identity/v3/users/c02531851c6c49c9abab77229f96d67f => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:01.617592 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59130d78-b9e2-4c55-8317-7c0185e85233 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:01.618263 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59130d78-b9e2-4c55-8317-7c0185e85233 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:01.618549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59130d78-b9e2-4c55-8317-7c0185e85233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:01.621276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b709da-ccaa-428e-99be-2d441f18bf11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:01.629605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-08b709da-ccaa-428e-99be-2d441f18bf11 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:01.629605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:01.629605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:01.629605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:01.629605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:01.629605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:01.629605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:01.629605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:01.629605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:01.629605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:01.629605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:01.629605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:01.629605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:01.629605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:01.629605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:01.629605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:01.630484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:01.630484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:01.630484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:01.630484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:01.630484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:01.630484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:01.630484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:01.630484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:01.630484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:01.630484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:01.630484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:01.630484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:01.630484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:01.630484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:01.630484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:01.630484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:01.631659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:01.631659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:01.631659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:01.631659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:01.631659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:01.631659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:01.631659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:01.631659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:01.631659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:01.631659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:01.631659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:01.631659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:01.631659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 62/122] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:01.632298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b709da-ccaa-428e-99be-2d441f18bf11 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:01.638962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-59130d78-b9e2-4c55-8317-7c0185e85233 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:01.640157 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-59130d78-b9e2-4c55-8317-7c0185e85233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:01.723536 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 61/123] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:01.728233 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:01.729644 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59130d78-b9e2-4c55-8317-7c0185e85233 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:01.752713 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:01.754165 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b709da-ccaa-428e-99be-2d441f18bf11 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:01.814864 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0e697-985e-48b0-a2d3-18ece6f72bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:01.816089 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0e697-985e-48b0-a2d3-18ece6f72bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:01.816439 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0e697-985e-48b0-a2d3-18ece6f72bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:01.816953 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0e697-985e-48b0-a2d3-18ece6f72bd8 None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:01.819030 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b0e697-985e-48b0-a2d3-18ece6f72bd8 None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:01.821155 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b0e697-985e-48b0-a2d3-18ece6f72bd8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:01.823773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 63/124] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:01 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:01.824623 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0e697-985e-48b0-a2d3-18ece6f72bd8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:01.867244 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59130d78-b9e2-4c55-8317-7c0185e85233 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:01.907301 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cedbd7-c0be-4e44-9121-dafef20413e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:01.909568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cedbd7-c0be-4e44-9121-dafef20413e1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:01.910072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cedbd7-c0be-4e44-9121-dafef20413e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:01.910541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cedbd7-c0be-4e44-9121-dafef20413e1 None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:01.912101 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cedbd7-c0be-4e44-9121-dafef20413e1 None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:01.913598 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cedbd7-c0be-4e44-9121-dafef20413e1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:01.916249 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 62/125] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:01 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:01.917024 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cedbd7-c0be-4e44-9121-dafef20413e1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:01.964221 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0e697-985e-48b0-a2d3-18ece6f72bd8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:02.006273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1f529-209e-4d7a-8965-fbb383dcb184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:02.007802 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1f529-209e-4d7a-8965-fbb383dcb184 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:02.008219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1f529-209e-4d7a-8965-fbb383dcb184 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:02.008632 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1f529-209e-4d7a-8965-fbb383dcb184 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:02.009787 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1f529-209e-4d7a-8965-fbb383dcb184 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:02.010270 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1f529-209e-4d7a-8965-fbb383dcb184 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:02.013683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1f529-209e-4d7a-8965-fbb383dcb184 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:02.019782 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:02.026980 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 64/126] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:02.028136 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1f529-209e-4d7a-8965-fbb383dcb184 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:02.050129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:02.053586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cedbd7-c0be-4e44-9121-dafef20413e1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:02.089206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e7713-e28d-45b1-99d2-901123bdb91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:02.089902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e7713-e28d-45b1-99d2-901123bdb91e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:02.090186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e7713-e28d-45b1-99d2-901123bdb91e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:02.090459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e7713-e28d-45b1-99d2-901123bdb91e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:02.100938 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e7713-e28d-45b1-99d2-901123bdb91e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:02.109393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e22e7713-e28d-45b1-99d2-901123bdb91e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:02.109393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:02.109393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:02.109393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:02.109393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:02.109393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:02.109393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:02.109393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:02.109393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:02.109393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:02.109393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:02.109393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:02.109393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:02.109393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:02.109393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:02.109393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:02.110354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:02.110354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:02.110354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:02.110354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:02.110354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:02.110354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:02.110354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:02.110354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:02.110354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:02.110354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:02.110354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:02.110354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:02.110354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:02.110354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:02.110354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:02.110354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:02.111189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:02.111189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:02.111189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:02.111189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:02.111189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:02.111189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:02.111189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:02.111189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:02.111189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:02.111189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:02.111189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:02.111189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:02.111189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 63/127] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:02.111813 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e7713-e28d-45b1-99d2-901123bdb91e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:02.166329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1f529-209e-4d7a-8965-fbb383dcb184 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:02.210347 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992a839-4614-43d2-9ed0-b7fde8b42b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:02.211278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992a839-4614-43d2-9ed0-b7fde8b42b7a None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:02.211563 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992a839-4614-43d2-9ed0-b7fde8b42b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:02.211840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992a839-4614-43d2-9ed0-b7fde8b42b7a None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be/users/c02531851c6c49c9abab77229f96d67f/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:02.214585 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b992a839-4614-43d2-9ed0-b7fde8b42b7a None admin] RBAC: Authorizing `identity:create_grant(project_id=aa7a11e47f684403b0ebf54d869da5be, user_id=c02531851c6c49c9abab77229f96d67f, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:02.216126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b992a839-4614-43d2-9ed0-b7fde8b42b7a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:02.234526 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 65/128] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:45:02 2022] PUT /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be/users/c02531851c6c49c9abab77229f96d67f/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:45:02.235126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e7713-e28d-45b1-99d2-901123bdb91e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:02.235611 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992a839-4614-43d2-9ed0-b7fde8b42b7a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:02.303893 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fee32-e988-4d96-ac12-d2cfa86df8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:02.305402 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fee32-e988-4d96-ac12-d2cfa86df8f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:02.305700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fee32-e988-4d96-ac12-d2cfa86df8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:02.307086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fee32-e988-4d96-ac12-d2cfa86df8f2 None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be/users/c02531851c6c49c9abab77229f96d67f/roles/662cb9b5dff94cdb83f292ad20d1d430` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:02.314599 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82fee32-e988-4d96-ac12-d2cfa86df8f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa7a11e47f684403b0ebf54d869da5be, user_id=c02531851c6c49c9abab77229f96d67f, role_id=662cb9b5dff94cdb83f292ad20d1d430)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:02.316105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82fee32-e988-4d96-ac12-d2cfa86df8f2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:02.326708 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:02.346182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 64/129] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:45:02 2022] PUT /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be/users/c02531851c6c49c9abab77229f96d67f/roles/662cb9b5dff94cdb83f292ad20d1d430 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:45:02.346833 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fee32-e988-4d96-ac12-d2cfa86df8f2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:02.401394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:02.463802 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992a839-4614-43d2-9ed0-b7fde8b42b7a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:02.532763 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612412c-5bca-497f-9b95-10e682c71527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:02.533670 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612412c-5bca-497f-9b95-10e682c71527 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:02.533964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612412c-5bca-497f-9b95-10e682c71527 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:02.534237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612412c-5bca-497f-9b95-10e682c71527 None admin] PATH_INFO: `/v3/users/81ad1712c9424afcafb9e80388158953` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:02.550975 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fee32-e988-4d96-ac12-d2cfa86df8f2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:02.566183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612412c-5bca-497f-9b95-10e682c71527 None admin] RBAC: Authorizing `identity:get_user(user_id=81ad1712c9424afcafb9e80388158953)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:02.567709 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612412c-5bca-497f-9b95-10e682c71527 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:02.569496 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 66/130] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:02 2022] GET /identity/v3/users/81ad1712c9424afcafb9e80388158953 => generated 289 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:02.577145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612412c-5bca-497f-9b95-10e682c71527 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:02.626613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c310da1-4813-4cc1-ac52-199ccf60642a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:02.627228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c310da1-4813-4cc1-ac52-199ccf60642a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:02.628794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c310da1-4813-4cc1-ac52-199ccf60642a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:02.629078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c310da1-4813-4cc1-ac52-199ccf60642a None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:02.629844 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c310da1-4813-4cc1-ac52-199ccf60642a None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:02.630154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c310da1-4813-4cc1-ac52-199ccf60642a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:02.631602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c310da1-4813-4cc1-ac52-199ccf60642a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:02.654749 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 65/131] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:02.660627 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c310da1-4813-4cc1-ac52-199ccf60642a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:02.781629 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612412c-5bca-497f-9b95-10e682c71527 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:02.783933 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:02.845674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:02.875460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c310da1-4813-4cc1-ac52-199ccf60642a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:02.877785 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1350265-f93b-4422-9e63-a5b028c774a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:02.878398 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1350265-f93b-4422-9e63-a5b028c774a6 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:02.878715 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1350265-f93b-4422-9e63-a5b028c774a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:02.879019 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1350265-f93b-4422-9e63-a5b028c774a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:02.943971 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a67af-0dfe-4320-8a8c-4b2397d14722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:02.944972 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a67af-0dfe-4320-8a8c-4b2397d14722 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:02.945258 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a67af-0dfe-4320-8a8c-4b2397d14722 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:02.949012 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a67af-0dfe-4320-8a8c-4b2397d14722 None admin] PATH_INFO: `/v3/users/c02531851c6c49c9abab77229f96d67f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:02.950810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a67af-0dfe-4320-8a8c-4b2397d14722 None admin] RBAC: Authorizing `identity:get_user(user_id=c02531851c6c49c9abab77229f96d67f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:02.953651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a67af-0dfe-4320-8a8c-4b2397d14722 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:02.956680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 66/132] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:02 2022] GET /identity/v3/users/c02531851c6c49c9abab77229f96d67f => generated 275 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:02.961929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a67af-0dfe-4320-8a8c-4b2397d14722 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:03.043874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1350265-f93b-4422-9e63-a5b028c774a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:03.045879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1350265-f93b-4422-9e63-a5b028c774a6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:03.116486 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a67af-0dfe-4320-8a8c-4b2397d14722 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:03.154849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 67/133] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:03.156543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8bb0d-3fb4-41c4-8bb6-b451541f6686 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:03.157033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8bb0d-3fb4-41c4-8bb6-b451541f6686 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:03.157526 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8bb0d-3fb4-41c4-8bb6-b451541f6686 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:03.159222 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 68/134] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:03.160425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8bb0d-3fb4-41c4-8bb6-b451541f6686 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:03.174378 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:03.185954 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb3b2f-dc60-47ef-83c1-09de5d665293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:03.187225 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb3b2f-dc60-47ef-83c1-09de5d665293 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:03.187712 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb3b2f-dc60-47ef-83c1-09de5d665293 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:03.188216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb3b2f-dc60-47ef-83c1-09de5d665293 None admin] PATH_INFO: `/v3/projects/42c0dc6e486e4990b8c551043685c92d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:03.189629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb3b2f-dc60-47ef-83c1-09de5d665293 None admin] RBAC: Authorizing `identity:get_project(project_id=42c0dc6e486e4990b8c551043685c92d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:03.191314 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb3b2f-dc60-47ef-83c1-09de5d665293 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:03.197951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 67/135] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:02 2022] GET /identity/v3/projects/42c0dc6e486e4990b8c551043685c92d => generated 312 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:03.200608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979616b-918c-4a8d-8ae4-1fccb03a0c4e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:03.201111 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979616b-918c-4a8d-8ae4-1fccb03a0c4e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:03.201397 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:03.201813 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979616b-918c-4a8d-8ae4-1fccb03a0c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:03.235957 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0979616b-918c-4a8d-8ae4-1fccb03a0c4e None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:03.238073 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0979616b-918c-4a8d-8ae4-1fccb03a0c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:03.297727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8bb0d-3fb4-41c4-8bb6-b451541f6686 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:03.365739 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b822f1-7328-499b-abcc-fcb4433a8c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:03.367006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b822f1-7328-499b-abcc-fcb4433a8c72 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:03.367515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b822f1-7328-499b-abcc-fcb4433a8c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:03.367994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b822f1-7328-499b-abcc-fcb4433a8c72 None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:03.369528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b822f1-7328-499b-abcc-fcb4433a8c72 None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:03.371887 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b822f1-7328-499b-abcc-fcb4433a8c72 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:03.375096 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 69/136] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:03 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:03.377104 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b822f1-7328-499b-abcc-fcb4433a8c72 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:03.378563 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 68/137] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:03.381785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979616b-918c-4a8d-8ae4-1fccb03a0c4e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:03.561667 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:03.568994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979616b-918c-4a8d-8ae4-1fccb03a0c4e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:03.574727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b822f1-7328-499b-abcc-fcb4433a8c72 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:03.616308 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:03.636461 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71fbfb-4923-40d2-9e75-6da0f02bffb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:03.637146 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71fbfb-4923-40d2-9e75-6da0f02bffb9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:03.637454 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71fbfb-4923-40d2-9e75-6da0f02bffb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:03.637770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71fbfb-4923-40d2-9e75-6da0f02bffb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:03.638644 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71fbfb-4923-40d2-9e75-6da0f02bffb9 None admin] RBAC: Adding query filter params (role.id=10a2c5c4144645358318a8aee7749f84, scope.project.id=aa7a11e47f684403b0ebf54d869da5be, user.id=81ad1712c9424afcafb9e80388158953) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:03.638983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71fbfb-4923-40d2-9e75-6da0f02bffb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:03.640466 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71fbfb-4923-40d2-9e75-6da0f02bffb9 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:03.645545 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136f8ec-f76a-4fd9-bac1-41755f905a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:03.646445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136f8ec-f76a-4fd9-bac1-41755f905a3d None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:03.646835 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136f8ec-f76a-4fd9-bac1-41755f905a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:03.647136 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136f8ec-f76a-4fd9-bac1-41755f905a3d None admin] PATH_INFO: `/v3/projects/42c0dc6e486e4990b8c551043685c92d/users/c02531851c6c49c9abab77229f96d67f/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:03.650174 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2136f8ec-f76a-4fd9-bac1-41755f905a3d None admin] RBAC: Authorizing `identity:create_grant(project_id=42c0dc6e486e4990b8c551043685c92d, user_id=c02531851c6c49c9abab77229f96d67f, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:03.651952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2136f8ec-f76a-4fd9-bac1-41755f905a3d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:03.657122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 69/138] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:03 2022] GET /identity/v3/role_assignments?user.id=81ad1712c9424afcafb9e80388158953&scope.project.id=aa7a11e47f684403b0ebf54d869da5be&role.id=10a2c5c4144645358318a8aee7749f84 => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:03.658455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71fbfb-4923-40d2-9e75-6da0f02bffb9 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:03.678203 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 70/139] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:45:03 2022] PUT /identity/v3/projects/42c0dc6e486e4990b8c551043685c92d/users/c02531851c6c49c9abab77229f96d67f/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:45:03.789357 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71fbfb-4923-40d2-9e75-6da0f02bffb9 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:03.828179 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12d8b2-1d28-4181-98a8-e8979bc3dc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:03.828924 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12d8b2-1d28-4181-98a8-e8979bc3dc00 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:03.829212 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12d8b2-1d28-4181-98a8-e8979bc3dc00 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:03.829496 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12d8b2-1d28-4181-98a8-e8979bc3dc00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:03.842283 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12d8b2-1d28-4181-98a8-e8979bc3dc00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:03.850153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-df12d8b2-1d28-4181-98a8-e8979bc3dc00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:45:03.850153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:03.850153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:03.850153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:03.850153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:03.850153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:03.850153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:03.850153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:03.850153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:03.850153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:03.850153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:03.850153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:03.850153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:03.850153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:03.850153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:03.850153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:03.851088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:03.851088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:03.851088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:03.851088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:03.851088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:03.851088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:03.851088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:03.851088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:03.851088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:03.851088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:03.851088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:03.851088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:03.851088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:03.851088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:03.851088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:03.851088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:03.852293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:03.852293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:03.852293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:03.852293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:03.852293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:03.852293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:03.852293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:03.852293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:03.852293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:03.852293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:03.852293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:45:03.852293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:03.852293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 70/140] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:45:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:03.857891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136f8ec-f76a-4fd9-bac1-41755f905a3d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:03.999145 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8be5e-0e66-470e-ada0-868eaa091ddf None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:03.999780 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8be5e-0e66-470e-ada0-868eaa091ddf None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:04.000297 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8be5e-0e66-470e-ada0-868eaa091ddf None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:04.001772 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 71/141] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:45:04.007399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dfb92-6f22-4163-814b-6596aebc9742 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:04.007813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136f8ec-f76a-4fd9-bac1-41755f905a3d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:04.008192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dfb92-6f22-4163-814b-6596aebc9742 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:04.008686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dfb92-6f22-4163-814b-6596aebc9742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:04.038369 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f61dfb92-6f22-4163-814b-6596aebc9742 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:04.039923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f61dfb92-6f22-4163-814b-6596aebc9742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:04.064182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e9a0b-8ea4-4ceb-8616-5c548cd5a4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:04.064891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e9a0b-8ea4-4ceb-8616-5c548cd5a4be None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:04.065231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e9a0b-8ea4-4ceb-8616-5c548cd5a4be None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:04.065520 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e9a0b-8ea4-4ceb-8616-5c548cd5a4be None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:04.066285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e9a0b-8ea4-4ceb-8616-5c548cd5a4be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:04.066679 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e9a0b-8ea4-4ceb-8616-5c548cd5a4be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:04.068340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e9a0b-8ea4-4ceb-8616-5c548cd5a4be None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:04.081351 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 71/142] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:04.087131 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e9a0b-8ea4-4ceb-8616-5c548cd5a4be None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:04.110988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:04.125346 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 72/143] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 16 18:45:04.127030 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:04.151206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dfb92-6f22-4163-814b-6596aebc9742 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:04.261962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dfb92-6f22-4163-814b-6596aebc9742 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:04.276214 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e9a0b-8ea4-4ceb-8616-5c548cd5a4be None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:04.300312 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a80c6-a001-4213-91f3-ca4377f6f90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:04.301483 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a80c6-a001-4213-91f3-ca4377f6f90f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:04.301966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a80c6-a001-4213-91f3-ca4377f6f90f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:04.302454 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a80c6-a001-4213-91f3-ca4377f6f90f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:04.303879 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a80c6-a001-4213-91f3-ca4377f6f90f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:04.305553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a80c6-a001-4213-91f3-ca4377f6f90f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:04.307465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 73/144] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:04.313733 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a80c6-a001-4213-91f3-ca4377f6f90f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:04.348505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e268f-3380-41f6-aec2-7e644fe12968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:04.349311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e268f-3380-41f6-aec2-7e644fe12968 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:04.349597 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e268f-3380-41f6-aec2-7e644fe12968 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:04.349873 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e268f-3380-41f6-aec2-7e644fe12968 None admin] PATH_INFO: `/v3/users/7e2576df31cd4791832f5441b6f98ba5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:04.351321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e268f-3380-41f6-aec2-7e644fe12968 None admin] RBAC: Authorizing `identity:get_user(user_id=7e2576df31cd4791832f5441b6f98ba5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:04.352995 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e268f-3380-41f6-aec2-7e644fe12968 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:04.355995 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 72/145] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:04 2022] GET /identity/v3/users/7e2576df31cd4791832f5441b6f98ba5 => generated 247 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:04.370881 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:04.372920 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e268f-3380-41f6-aec2-7e644fe12968 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:04.429196 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a80c6-a001-4213-91f3-ca4377f6f90f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:04.466684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa06ef-2cba-4695-a44f-b199d89ea310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:04.467610 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa06ef-2cba-4695-a44f-b199d89ea310 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:04.468038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa06ef-2cba-4695-a44f-b199d89ea310 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:04.468458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa06ef-2cba-4695-a44f-b199d89ea310 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:04.469320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa06ef-2cba-4695-a44f-b199d89ea310 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:04.471525 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa06ef-2cba-4695-a44f-b199d89ea310 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:04.503203 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 74/146] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:45:04 2022] POST /identity/v3/users => generated 283 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:45:04.548979 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:04.566917 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e268f-3380-41f6-aec2-7e644fe12968 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:04.610877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93816a-507c-4b51-9c75-d7c8904e2586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:04.611528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93816a-507c-4b51-9c75-d7c8904e2586 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:04.611820 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93816a-507c-4b51-9c75-d7c8904e2586 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:04.612102 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93816a-507c-4b51-9c75-d7c8904e2586 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:04.688300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93816a-507c-4b51-9c75-d7c8904e2586 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:04.689794 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93816a-507c-4b51-9c75-d7c8904e2586 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:04.783191 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:04.797995 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 73/147] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:45:04.803840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa06ef-2cba-4695-a44f-b199d89ea310 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:04.919327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa06ef-2cba-4695-a44f-b199d89ea310 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:04.921173 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:04.964924 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d3a3b-52d1-4083-954a-c5399af1f172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:04.965820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d3a3b-52d1-4083-954a-c5399af1f172 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:04.966111 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d3a3b-52d1-4083-954a-c5399af1f172 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:04.966397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d3a3b-52d1-4083-954a-c5399af1f172 None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:04.971310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d3a3b-52d1-4083-954a-c5399af1f172 None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:04.972791 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d3a3b-52d1-4083-954a-c5399af1f172 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:04.974516 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 75/148] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:04 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:04.981360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93816a-507c-4b51-9c75-d7c8904e2586 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:05.110629 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93816a-507c-4b51-9c75-d7c8904e2586 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:05.151503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c10855-459d-454f-840d-04ac8b7fc896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:05.152182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c10855-459d-454f-840d-04ac8b7fc896 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:05.152464 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c10855-459d-454f-840d-04ac8b7fc896 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:05.152742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c10855-459d-454f-840d-04ac8b7fc896 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:05.153583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c10855-459d-454f-840d-04ac8b7fc896 None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=aa7a11e47f684403b0ebf54d869da5be, user.id=7e2576df31cd4791832f5441b6f98ba5) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:05.153895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c10855-459d-454f-840d-04ac8b7fc896 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:05.155379 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c10855-459d-454f-840d-04ac8b7fc896 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:05.167932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 74/149] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:04 2022] GET /identity/v3/role_assignments?user.id=7e2576df31cd4791832f5441b6f98ba5&scope.project.id=aa7a11e47f684403b0ebf54d869da5be&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:05.396524 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e88c6-635e-44fb-a55c-5ca08cc29d92 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:05.397028 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e88c6-635e-44fb-a55c-5ca08cc29d92 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:05.397100 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e88c6-635e-44fb-a55c-5ca08cc29d92 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:05.398687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 76/150] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:05.404809 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e7756-e678-435c-91a1-837a769f8e3a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:05.405367 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e7756-e678-435c-91a1-837a769f8e3a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:05.405845 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e7756-e678-435c-91a1-837a769f8e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:05.424119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-6b7e7756-e678-435c-91a1-837a769f8e3a None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:05.425640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6b7e7756-e678-435c-91a1-837a769f8e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:05.483900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:05.510758 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 75/151] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:05.536775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e88c6-635e-44fb-a55c-5ca08cc29d92 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:05.652373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e88c6-635e-44fb-a55c-5ca08cc29d92 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:05.692572 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96cda0-47a5-4d49-a7d6-a361a1cdcdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:05.693436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96cda0-47a5-4d49-a7d6-a361a1cdcdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:05.693725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96cda0-47a5-4d49-a7d6-a361a1cdcdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:05.694001 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96cda0-47a5-4d49-a7d6-a361a1cdcdd4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:05.706692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96cda0-47a5-4d49-a7d6-a361a1cdcdd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:05.715430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-9f96cda0-47a5-4d49-a7d6-a361a1cdcdd4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:05.715430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:05.715430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:05.715430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:05.715430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:05.715430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:05.715430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:05.715430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:05.715430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:05.715430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:05.715430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:05.715430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:05.715430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:05.715430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:05.715430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:05.715430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:05.716456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:05.716456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:05.716456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:05.716456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:05.716456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:05.716456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:05.716456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:05.716456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:05.716456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:05.716456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:05.716456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:05.716456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:05.716456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:05.716456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:05.716456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:05.716456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:05.717221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:05.717221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:05.717221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:05.717221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:05.717221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:05.717221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:05.717221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:05.717221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:05.717221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:05.717221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:05.717221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:05.717221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:05.722891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 77/152] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:05.725178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e7756-e678-435c-91a1-837a769f8e3a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:05.859110 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e7756-e678-435c-91a1-837a769f8e3a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:05.871002 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f4846-d241-476d-bfda-9433d7eb148e None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:05.871671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f4846-d241-476d-bfda-9433d7eb148e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:05.872201 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f4846-d241-476d-bfda-9433d7eb148e None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:05.873869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 78/153] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:05.881344 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08cd9a-3f6d-4f64-8fc2-d6c97140fdf3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:05.881888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08cd9a-3f6d-4f64-8fc2-d6c97140fdf3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:05.882363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08cd9a-3f6d-4f64-8fc2-d6c97140fdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:05.919051 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7f08cd9a-3f6d-4f64-8fc2-d6c97140fdf3 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:05.920641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7f08cd9a-3f6d-4f64-8fc2-d6c97140fdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:05.939307 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e192c-24c4-44e9-b008-3285a89080af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:05.940001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e192c-24c4-44e9-b008-3285a89080af None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:05.940282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e192c-24c4-44e9-b008-3285a89080af None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:05.940554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e192c-24c4-44e9-b008-3285a89080af None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:05.941308 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e192c-24c4-44e9-b008-3285a89080af None admin] RBAC: Adding query filter params (name=member) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:05.941616 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e192c-24c4-44e9-b008-3285a89080af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:05.943017 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e192c-24c4-44e9-b008-3285a89080af None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:05.968840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 76/154] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:05.979062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e192c-24c4-44e9-b008-3285a89080af None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:05.980583 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:06.034187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 79/155] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:06.050523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:06.068768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08cd9a-3f6d-4f64-8fc2-d6c97140fdf3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:06.155019 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e192c-24c4-44e9-b008-3285a89080af None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:06.228163 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08cd9a-3f6d-4f64-8fc2-d6c97140fdf3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:06.252260 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd522c8a-9a05-4aa9-8117-79bb9e15fb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:06.253161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd522c8a-9a05-4aa9-8117-79bb9e15fb02 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:06.253439 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd522c8a-9a05-4aa9-8117-79bb9e15fb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:06.253901 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd522c8a-9a05-4aa9-8117-79bb9e15fb02 None admin] PATH_INFO: `/v3/users/c02531851c6c49c9abab77229f96d67f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:06.255282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd522c8a-9a05-4aa9-8117-79bb9e15fb02 None admin] RBAC: Authorizing `identity:get_user(user_id=c02531851c6c49c9abab77229f96d67f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:06.256870 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd522c8a-9a05-4aa9-8117-79bb9e15fb02 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:06.258966 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 77/156] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:05 2022] GET /identity/v3/users/c02531851c6c49c9abab77229f96d67f => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:06.271393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bdcdc-5d8a-4775-86c9-1f9e71c07afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:06.272218 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bdcdc-5d8a-4775-86c9-1f9e71c07afe None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:06.272503 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bdcdc-5d8a-4775-86c9-1f9e71c07afe None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:06.272780 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bdcdc-5d8a-4775-86c9-1f9e71c07afe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:06.274548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd522c8a-9a05-4aa9-8117-79bb9e15fb02 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:06.284980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238bdcdc-5d8a-4775-86c9-1f9e71c07afe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:06.295062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-238bdcdc-5d8a-4775-86c9-1f9e71c07afe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:06.295062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:06.295062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:06.295062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:06.295062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:06.295062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:06.295062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:06.295062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:06.295062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:06.295062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:06.295062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:06.295062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:06.295062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:06.295062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:06.295062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:06.295062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:06.296303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:06.296303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:06.296303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:06.296303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:06.296303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:06.296303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:06.296303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:06.296303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:06.296303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:06.296303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:06.296303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:06.296303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:06.296303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:06.296303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:06.296303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:06.296303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:06.297345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:06.297345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:06.297345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:06.297345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:06.297345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:06.297345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:06.297345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:06.297345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:06.297345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:06.297345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:06.297345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:06.297345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:06.302687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 80/157] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:45:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:06.307631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bdcdc-5d8a-4775-86c9-1f9e71c07afe None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:06.366951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:06.580869 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd522c8a-9a05-4aa9-8117-79bb9e15fb02 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:06.596727 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bdcdc-5d8a-4775-86c9-1f9e71c07afe None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:06.602903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:06.680258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7aaa8-699d-4d05-8995-68073fdbbaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:06.680934 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7aaa8-699d-4d05-8995-68073fdbbaea None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:06.681220 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7aaa8-699d-4d05-8995-68073fdbbaea None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:06.681496 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7aaa8-699d-4d05-8995-68073fdbbaea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:06.699330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f8902-4759-4698-9cd1-505429112ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:06.700103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f8902-4759-4698-9cd1-505429112ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:06.700385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f8902-4759-4698-9cd1-505429112ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:06.700661 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f8902-4759-4698-9cd1-505429112ea3 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:06.701426 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f8902-4759-4698-9cd1-505429112ea3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:06.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f8902-4759-4698-9cd1-505429112ea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:06.703162 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f8902-4759-4698-9cd1-505429112ea3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:06.717901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 81/158] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:45:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:06.723831 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f8902-4759-4698-9cd1-505429112ea3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:06.844478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7aaa8-699d-4d05-8995-68073fdbbaea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:06.845873 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7aaa8-699d-4d05-8995-68073fdbbaea None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:06.914938 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:06.958860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f8902-4759-4698-9cd1-505429112ea3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:07.031604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffa5ee-4cd8-4ae6-9007-fa175abc3bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:07.032507 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffa5ee-4cd8-4ae6-9007-fa175abc3bce None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:07.032791 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffa5ee-4cd8-4ae6-9007-fa175abc3bce None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:07.033066 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffa5ee-4cd8-4ae6-9007-fa175abc3bce None admin] PATH_INFO: `/v3/users/c02531851c6c49c9abab77229f96d67f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:07.035496 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffa5ee-4cd8-4ae6-9007-fa175abc3bce None admin] RBAC: Authorizing `identity:get_user(user_id=c02531851c6c49c9abab77229f96d67f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:07.036954 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffa5ee-4cd8-4ae6-9007-fa175abc3bce None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:07.038639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 82/159] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:06 2022] GET /identity/v3/users/c02531851c6c49c9abab77229f96d67f => generated 275 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:07.045741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffa5ee-4cd8-4ae6-9007-fa175abc3bce None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:07.101565 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 78/160] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 827 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:07.108339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7aaa8-699d-4d05-8995-68073fdbbaea None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:07.136842 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:07.167347 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffa5ee-4cd8-4ae6-9007-fa175abc3bce None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:07.208835 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ada2e-4e88-41b1-96cb-dead90c8c934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:07.209514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ada2e-4e88-41b1-96cb-dead90c8c934 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:07.209799 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ada2e-4e88-41b1-96cb-dead90c8c934 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:07.210077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ada2e-4e88-41b1-96cb-dead90c8c934 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:07.246984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:07.300070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ada2e-4e88-41b1-96cb-dead90c8c934 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:07.301747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ada2e-4e88-41b1-96cb-dead90c8c934 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:07.350248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7aaa8-699d-4d05-8995-68073fdbbaea None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:07.425016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d74205-e603-497b-b7a0-2540853a834b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:07.425813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d74205-e603-497b-b7a0-2540853a834b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:07.426097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d74205-e603-497b-b7a0-2540853a834b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:07.426373 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d74205-e603-497b-b7a0-2540853a834b None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:07.428794 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d74205-e603-497b-b7a0-2540853a834b None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:07.430264 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d74205-e603-497b-b7a0-2540853a834b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:07.431987 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 79/161] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:07 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:07.440840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8899f-69bc-4955-af41-adfb9e9cd9e8 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:07.441130 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8899f-69bc-4955-af41-adfb9e9cd9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:07.441415 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8899f-69bc-4955-af41-adfb9e9cd9e8 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:07.444482 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 80/162] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:07 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:07.445643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9ba99-2110-4938-9e7d-f85bdacf3820 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:07.445927 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9ba99-2110-4938-9e7d-f85bdacf3820 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:07.446204 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9ba99-2110-4938-9e7d-f85bdacf3820 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:07.451753 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 81/163] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:07.452329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9ba99-2110-4938-9e7d-f85bdacf3820 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:07.483729 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 83/164] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:45:07.485367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95576a0d-cb2f-4bf8-b0dc-9896b4c3e27c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:07.485798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95576a0d-cb2f-4bf8-b0dc-9896b4c3e27c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:07.486211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95576a0d-cb2f-4bf8-b0dc-9896b4c3e27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:07.505633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-95576a0d-cb2f-4bf8-b0dc-9896b4c3e27c None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:07.507233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-95576a0d-cb2f-4bf8-b0dc-9896b4c3e27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:07.581432 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 84/165] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:07.583110 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78049b-ad00-4bac-b475-2f1bb3cb0129 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:07.583820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78049b-ad00-4bac-b475-2f1bb3cb0129 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:07.584239 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78049b-ad00-4bac-b475-2f1bb3cb0129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:07.598912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:07.614958 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4b78049b-ad00-4bac-b475-2f1bb3cb0129 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:07.616382 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4b78049b-ad00-4bac-b475-2f1bb3cb0129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:07.654912 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:07.694832 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 85/166] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:07.695942 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78049b-ad00-4bac-b475-2f1bb3cb0129 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:07.725456 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9ba99-2110-4938-9e7d-f85bdacf3820 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:07.791910 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a261079-d287-44ee-b0f7-e0f1c953781d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:07.793579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a261079-d287-44ee-b0f7-e0f1c953781d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:07.794068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a261079-d287-44ee-b0f7-e0f1c953781d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:07.794534 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a261079-d287-44ee-b0f7-e0f1c953781d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:07.795621 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a261079-d287-44ee-b0f7-e0f1c953781d None admin] RBAC: Adding query filter params (role.id=cf55e7c6398f45efaf10dec4fe153f8a, scope.project.id=aa7a11e47f684403b0ebf54d869da5be, user.id=c02531851c6c49c9abab77229f96d67f) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:07.796127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a261079-d287-44ee-b0f7-e0f1c953781d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:07.797761 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a261079-d287-44ee-b0f7-e0f1c953781d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:07.810826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 82/167] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:07 2022] GET /identity/v3/role_assignments?user.id=c02531851c6c49c9abab77229f96d67f&scope.project.id=aa7a11e47f684403b0ebf54d869da5be&role.id=cf55e7c6398f45efaf10dec4fe153f8a => generated 608 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:07.812954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a261079-d287-44ee-b0f7-e0f1c953781d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:07.884653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78049b-ad00-4bac-b475-2f1bb3cb0129 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:07.900736 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:07.922111 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2fa9e-8666-4571-b7c2-b58cb8536b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:07.923033 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2fa9e-8666-4571-b7c2-b58cb8536b54 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:07.923315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2fa9e-8666-4571-b7c2-b58cb8536b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:07.923588 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2fa9e-8666-4571-b7c2-b58cb8536b54 None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:07.924888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2fa9e-8666-4571-b7c2-b58cb8536b54 None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:07.926380 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2fa9e-8666-4571-b7c2-b58cb8536b54 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:07.928278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 86/168] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:07 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:07.929127 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2fa9e-8666-4571-b7c2-b58cb8536b54 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:08.037979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2fa9e-8666-4571-b7c2-b58cb8536b54 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:08.074220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a420b8-391e-4c94-ade2-c87c775c88b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:08.075123 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a420b8-391e-4c94-ade2-c87c775c88b2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:08.075406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a420b8-391e-4c94-ade2-c87c775c88b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:08.075675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a420b8-391e-4c94-ade2-c87c775c88b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:08.086909 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a420b8-391e-4c94-ade2-c87c775c88b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:08.095387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-15a420b8-391e-4c94-ade2-c87c775c88b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:08.095387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:08.095387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:08.095387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:08.095387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:08.095387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:08.095387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:08.095387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:08.095387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:08.095387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:08.095387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:08.095387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:08.095387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:08.095387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:08.095387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:08.095387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:08.096520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:08.096520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:08.096520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:08.096520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:08.096520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:08.096520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:08.096520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:08.096520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:08.096520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:08.096520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:08.096520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:08.096520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:08.096520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:08.096520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:08.096520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:08.096520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:08.097338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:08.097338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:08.097338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:08.097338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:08.097338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:08.097338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:08.097338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:08.097338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:08.097338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:08.097338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:08.097338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:08.097338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:08.097338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 87/169] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:08.097982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a420b8-391e-4c94-ade2-c87c775c88b2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:08.134617 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a261079-d287-44ee-b0f7-e0f1c953781d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:08.141919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:08.194758 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b6a8d-fd98-4994-940f-618512d0f1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:08.195559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b6a8d-fd98-4994-940f-618512d0f1b4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:08.196000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b6a8d-fd98-4994-940f-618512d0f1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:08.196411 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b6a8d-fd98-4994-940f-618512d0f1b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:08.210984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b6a8d-fd98-4994-940f-618512d0f1b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:08.221315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ce2b6a8d-fd98-4994-940f-618512d0f1b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:08.221315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:08.221315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:08.221315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:08.221315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:08.221315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:08.221315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:08.221315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:08.221315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:08.221315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:08.221315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:08.221315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:08.221315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:08.221315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:08.221315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:08.221315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:08.222409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:08.222409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:08.222409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:08.222409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:08.222409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:08.222409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:08.222409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:08.222409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:08.222409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:08.222409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:08.222409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:08.222409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:08.222409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:08.222409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:08.222409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:08.222409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:08.223492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:08.223492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:08.223492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:08.223492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:08.223492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:08.223492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:08.223492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:08.223492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:08.223492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:08.223492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:08.223492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:08.223492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:08.225718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 83/170] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:08.226908 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b6a8d-fd98-4994-940f-618512d0f1b4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:08.230594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:08.322384 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a420b8-391e-4c94-ade2-c87c775c88b2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:08.351707 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b6a8d-fd98-4994-940f-618512d0f1b4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:08.390284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0332311-9d95-4b1d-a3f7-bc6961359b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:08.391007 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0332311-9d95-4b1d-a3f7-bc6961359b66 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:08.391633 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0332311-9d95-4b1d-a3f7-bc6961359b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:08.391738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0332311-9d95-4b1d-a3f7-bc6961359b66 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:08.392431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0332311-9d95-4b1d-a3f7-bc6961359b66 None admin] RBAC: Adding query filter params (name=member) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:08.392748 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0332311-9d95-4b1d-a3f7-bc6961359b66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:08.394404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0332311-9d95-4b1d-a3f7-bc6961359b66 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:08.404207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 84/171] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:08.404986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0332311-9d95-4b1d-a3f7-bc6961359b66 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:08.416210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7808dc3-7c4d-4e91-ba0b-c8fd12068482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:08.416953 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7808dc3-7c4d-4e91-ba0b-c8fd12068482 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:08.417894 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7808dc3-7c4d-4e91-ba0b-c8fd12068482 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:08.418184 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7808dc3-7c4d-4e91-ba0b-c8fd12068482 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:08.419088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7808dc3-7c4d-4e91-ba0b-c8fd12068482 None admin] RBAC: Adding query filter params (role.id=662cb9b5dff94cdb83f292ad20d1d430, scope.project.id=aa7a11e47f684403b0ebf54d869da5be, user.id=c02531851c6c49c9abab77229f96d67f) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:08.419407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7808dc3-7c4d-4e91-ba0b-c8fd12068482 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:08.420900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7808dc3-7c4d-4e91-ba0b-c8fd12068482 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:08.437838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 88/172] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:08 2022] GET /identity/v3/role_assignments?user.id=c02531851c6c49c9abab77229f96d67f&scope.project.id=aa7a11e47f684403b0ebf54d869da5be&role.id=662cb9b5dff94cdb83f292ad20d1d430 => generated 608 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:08.442726 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7808dc3-7c4d-4e91-ba0b-c8fd12068482 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:08.501924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:08.550147 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0332311-9d95-4b1d-a3f7-bc6961359b66 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:08.611801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0b63c-bdcb-4fa2-9a59-4910d453b328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:08.612766 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0b63c-bdcb-4fa2-9a59-4910d453b328 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:08.613197 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0b63c-bdcb-4fa2-9a59-4910d453b328 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:08.613620 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0b63c-bdcb-4fa2-9a59-4910d453b328 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:08.614673 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0b63c-bdcb-4fa2-9a59-4910d453b328 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:08.615145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0b63c-bdcb-4fa2-9a59-4910d453b328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:08.616804 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0b63c-bdcb-4fa2-9a59-4910d453b328 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:08.631987 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 85/173] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:08.638454 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0b63c-bdcb-4fa2-9a59-4910d453b328 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:08.778322 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7808dc3-7c4d-4e91-ba0b-c8fd12068482 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:08.779533 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0b63c-bdcb-4fa2-9a59-4910d453b328 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:08.783006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:08.820329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:08.832229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de478cc-60e1-448d-ae98-4f63a216de68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:08.833160 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de478cc-60e1-448d-ae98-4f63a216de68 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:08.833545 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de478cc-60e1-448d-ae98-4f63a216de68 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:08.833823 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de478cc-60e1-448d-ae98-4f63a216de68 None admin] PATH_INFO: `/v3/users/81ad1712c9424afcafb9e80388158953` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:08.835133 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de478cc-60e1-448d-ae98-4f63a216de68 None admin] RBAC: Authorizing `identity:get_user(user_id=81ad1712c9424afcafb9e80388158953)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:08.836646 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de478cc-60e1-448d-ae98-4f63a216de68 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:08.838431 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862bb77d-89ee-44de-851b-225f19205501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:08.838635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 86/174] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:08 2022] GET /identity/v3/users/81ad1712c9424afcafb9e80388158953 => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:08.840060 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bb77d-89ee-44de-851b-225f19205501 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:08.840176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b961df-583b-46b5-a3d2-5965d32ed4d0 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:08.840336 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b961df-583b-46b5-a3d2-5965d32ed4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:08.840440 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bb77d-89ee-44de-851b-225f19205501 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:08.840614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b961df-583b-46b5-a3d2-5965d32ed4d0 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:08.840709 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bb77d-89ee-44de-851b-225f19205501 None admin] PATH_INFO: `/v3/users/c02531851c6c49c9abab77229f96d67f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:08.841834 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 87/175] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:08.842188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862bb77d-89ee-44de-851b-225f19205501 None admin] RBAC: Authorizing `identity:get_user(user_id=c02531851c6c49c9abab77229f96d67f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:08.843803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862bb77d-89ee-44de-851b-225f19205501 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:08.845950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 89/176] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:08 2022] GET /identity/v3/users/c02531851c6c49c9abab77229f96d67f => generated 275 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:08.846369 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b961df-583b-46b5-a3d2-5965d32ed4d0 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:08.851555 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511a6eb-a4a3-41a5-8973-021fabdf3190 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:08.851847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511a6eb-a4a3-41a5-8973-021fabdf3190 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:08.852130 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511a6eb-a4a3-41a5-8973-021fabdf3190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:08.875272 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1511a6eb-a4a3-41a5-8973-021fabdf3190 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:08.876561 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1511a6eb-a4a3-41a5-8973-021fabdf3190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:08.963404 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 90/177] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:08.965160 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511a6eb-a4a3-41a5-8973-021fabdf3190 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:08.997331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b961df-583b-46b5-a3d2-5965d32ed4d0 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:09.044237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3a41a-25f9-4054-8fba-ed2063aa7273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:09.045037 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3a41a-25f9-4054-8fba-ed2063aa7273 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:09.045317 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3a41a-25f9-4054-8fba-ed2063aa7273 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:09.045627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3a41a-25f9-4054-8fba-ed2063aa7273 None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:09.046843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3a41a-25f9-4054-8fba-ed2063aa7273 None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:09.048285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3a41a-25f9-4054-8fba-ed2063aa7273 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:09.050166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 88/178] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:08 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:09.051187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1347d21-23f2-4144-98a9-5cc3ad01ec8a None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:09.051488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1347d21-23f2-4144-98a9-5cc3ad01ec8a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:09.051738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1347d21-23f2-4144-98a9-5cc3ad01ec8a None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:09.053072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 89/179] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:09.058705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1347d21-23f2-4144-98a9-5cc3ad01ec8a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:09.088383 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:09.091624 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511a6eb-a4a3-41a5-8973-021fabdf3190 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:09.128470 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450d875-b3a5-44e4-a59c-f9d04525d062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:09.129132 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450d875-b3a5-44e4-a59c-f9d04525d062 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:09.129419 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450d875-b3a5-44e4-a59c-f9d04525d062 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:09.129696 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450d875-b3a5-44e4-a59c-f9d04525d062 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:09.129944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:09.176860 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1347d21-23f2-4144-98a9-5cc3ad01ec8a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:09.212598 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450d875-b3a5-44e4-a59c-f9d04525d062 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:09.213128 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9dc18-1641-4650-9799-4ebd4700e516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:09.214709 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9dc18-1641-4650-9799-4ebd4700e516 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:09.215001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9dc18-1641-4650-9799-4ebd4700e516 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:09.215281 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9dc18-1641-4650-9799-4ebd4700e516 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:09.215868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450d875-b3a5-44e4-a59c-f9d04525d062 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:09.228070 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9dc18-1641-4650-9799-4ebd4700e516 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:09.236015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-74a9dc18-1641-4650-9799-4ebd4700e516 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:45:09.236015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:09.236015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:09.236015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:09.236015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:09.236015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:09.236015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:09.236015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:09.236015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:09.236015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:09.236015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:09.236015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:09.236015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:09.236015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:09.236015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:09.236015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:09.236943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:09.236943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:09.236943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:09.236943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:09.236943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:09.236943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:09.236943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:09.236943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:09.236943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:09.236943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:09.236943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:09.236943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:09.236943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:09.236943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:09.236943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:09.236943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:09.237724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:09.237724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:09.237724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:09.237724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:09.237724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:09.237724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:09.237724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:09.237724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:09.237724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:09.237724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:09.237724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:45:09.237724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:09.237724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 90/180] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:45:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:09.239069 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9dc18-1641-4650-9799-4ebd4700e516 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:09.324422 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:09.328118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 91/181] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:09.329609 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652ead0-1798-4df7-91e5-c688cd49b2be None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:09.329903 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652ead0-1798-4df7-91e5-c688cd49b2be None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:09.330415 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652ead0-1798-4df7-91e5-c688cd49b2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:09.344944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9dc18-1641-4650-9799-4ebd4700e516 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:09.345726 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e652ead0-1798-4df7-91e5-c688cd49b2be None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:09.346938 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e652ead0-1798-4df7-91e5-c688cd49b2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:09.367408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:09.379757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd5563-b943-4819-9c66-dd191bdb00f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:09.380642 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd5563-b943-4819-9c66-dd191bdb00f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:09.380921 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd5563-b943-4819-9c66-dd191bdb00f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:09.381195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd5563-b943-4819-9c66-dd191bdb00f6 None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be/users/81ad1712c9424afcafb9e80388158953/roles/10a2c5c4144645358318a8aee7749f84` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:09.383186 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd5563-b943-4819-9c66-dd191bdb00f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa7a11e47f684403b0ebf54d869da5be, user_id=81ad1712c9424afcafb9e80388158953, role_id=10a2c5c4144645358318a8aee7749f84)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:09.384615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd5563-b943-4819-9c66-dd191bdb00f6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:09.397775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 91/182] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:45:09 2022] PUT /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be/users/81ad1712c9424afcafb9e80388158953/roles/10a2c5c4144645358318a8aee7749f84 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:45:09.398345 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd5563-b943-4819-9c66-dd191bdb00f6 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:09.423288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 92/183] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:09.429036 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c50d45-8f85-4420-a81e-ce2edc2675a0 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:09.429941 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c50d45-8f85-4420-a81e-ce2edc2675a0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:09.430446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c50d45-8f85-4420-a81e-ce2edc2675a0 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:09.433118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 93/184] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:09.437779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c50d45-8f85-4420-a81e-ce2edc2675a0 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:09.521039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd5563-b943-4819-9c66-dd191bdb00f6 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:09.565150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c90c77-dc60-48e4-863d-3f5631062254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:09.565836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c90c77-dc60-48e4-863d-3f5631062254 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:09.566103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c90c77-dc60-48e4-863d-3f5631062254 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:09.566376 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c90c77-dc60-48e4-863d-3f5631062254 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:09.567173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c90c77-dc60-48e4-863d-3f5631062254 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:09.567480 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c90c77-dc60-48e4-863d-3f5631062254 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:09.568524 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c50d45-8f85-4420-a81e-ce2edc2675a0 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:09.568972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c90c77-dc60-48e4-863d-3f5631062254 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:09.578288 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 92/185] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:09.579542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edfbd6-a31e-46f8-814b-193ee03c3a0c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:09.579882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edfbd6-a31e-46f8-814b-193ee03c3a0c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:09.580103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edfbd6-a31e-46f8-814b-193ee03c3a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:09.597378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f4edfbd6-a31e-46f8-814b-193ee03c3a0c None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:09.598640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f4edfbd6-a31e-46f8-814b-193ee03c3a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:09.600585 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:09.607844 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300406f-f6e4-491d-8dd9-a9dd537f12db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:09.609193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300406f-f6e4-491d-8dd9-a9dd537f12db None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:09.609492 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300406f-f6e4-491d-8dd9-a9dd537f12db None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:09.609773 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300406f-f6e4-491d-8dd9-a9dd537f12db None admin] PATH_INFO: `/v3/projects/42c0dc6e486e4990b8c551043685c92d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:09.611617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300406f-f6e4-491d-8dd9-a9dd537f12db None admin] RBAC: Authorizing `identity:get_project(project_id=42c0dc6e486e4990b8c551043685c92d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:09.613989 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300406f-f6e4-491d-8dd9-a9dd537f12db None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:09.615933 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 94/186] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:09 2022] GET /identity/v3/projects/42c0dc6e486e4990b8c551043685c92d => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:09.616626 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300406f-f6e4-491d-8dd9-a9dd537f12db None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:09.647730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:09.686609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 93/187] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:09.686609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edfbd6-a31e-46f8-814b-193ee03c3a0c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:09.755576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300406f-f6e4-491d-8dd9-a9dd537f12db None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:09.785826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edfbd6-a31e-46f8-814b-193ee03c3a0c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:09.797039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc97bd1-3728-4291-9a8b-83e84230c653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:09.797983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc97bd1-3728-4291-9a8b-83e84230c653 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:09.798268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc97bd1-3728-4291-9a8b-83e84230c653 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:09.798546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc97bd1-3728-4291-9a8b-83e84230c653 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:09.809878 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc97bd1-3728-4291-9a8b-83e84230c653 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:09.818986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-0cc97bd1-3728-4291-9a8b-83e84230c653 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:09.818986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:09.818986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:09.818986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:09.818986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:09.818986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:09.818986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:09.818986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:09.818986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:09.818986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:09.818986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:09.818986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:09.818986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:09.818986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:09.818986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:09.818986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:09.819876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:09.819876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:09.819876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:09.819876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:09.819876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:09.819876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:09.819876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:09.819876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:09.819876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:09.819876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:09.819876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:09.819876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:09.819876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:09.819876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:09.819876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:09.819876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:09.820650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:09.820650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:09.820650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:09.820650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:09.820650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:09.820650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:09.820650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:09.820650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:09.820650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:09.820650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:09.820650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:09.820650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:09.820650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 95/188] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:45:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:09.821389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc97bd1-3728-4291-9a8b-83e84230c653 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:09.825865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebfaca-be2c-48f2-b555-c5c01044e4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:09.826779 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebfaca-be2c-48f2-b555-c5c01044e4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:09.827063 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebfaca-be2c-48f2-b555-c5c01044e4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:09.827339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebfaca-be2c-48f2-b555-c5c01044e4d9 None admin] PATH_INFO: `/v3/users/46d60b27f67a4955acb638a969d452ea` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:09.841612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eebfaca-be2c-48f2-b555-c5c01044e4d9 None admin] RBAC: Authorizing `identity:get_user(user_id=46d60b27f67a4955acb638a969d452ea)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:09.843093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eebfaca-be2c-48f2-b555-c5c01044e4d9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:09.844808 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 94/189] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:09 2022] GET /identity/v3/users/46d60b27f67a4955acb638a969d452ea => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:09.845569 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebfaca-be2c-48f2-b555-c5c01044e4d9 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:09.882858 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:09.896730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:09.957485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc97bd1-3728-4291-9a8b-83e84230c653 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:09.970091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebfaca-be2c-48f2-b555-c5c01044e4d9 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:09.993211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c3b30-1e7d-46ac-b7a5-40025e0d770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:09.993875 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c3b30-1e7d-46ac-b7a5-40025e0d770c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:09.994163 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c3b30-1e7d-46ac-b7a5-40025e0d770c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:09.994444 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c3b30-1e7d-46ac-b7a5-40025e0d770c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:09.996590 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c3b30-1e7d-46ac-b7a5-40025e0d770c None admin] RBAC: Adding query filter params (role.id=cf55e7c6398f45efaf10dec4fe153f8a, scope.project.id=42c0dc6e486e4990b8c551043685c92d, user.id=c02531851c6c49c9abab77229f96d67f) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:09.996907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c3b30-1e7d-46ac-b7a5-40025e0d770c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:09.998369 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c3b30-1e7d-46ac-b7a5-40025e0d770c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:10.004569 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 96/190] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:09 2022] GET /identity/v3/role_assignments?user.id=c02531851c6c49c9abab77229f96d67f&scope.project.id=42c0dc6e486e4990b8c551043685c92d&role.id=cf55e7c6398f45efaf10dec4fe153f8a => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:10.005199 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c3b30-1e7d-46ac-b7a5-40025e0d770c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:10.017059 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10568250-2e18-4165-b82e-2603c340e2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:10.017988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10568250-2e18-4165-b82e-2603c340e2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:10.018341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10568250-2e18-4165-b82e-2603c340e2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:10.018761 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10568250-2e18-4165-b82e-2603c340e2c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:10.020383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10568250-2e18-4165-b82e-2603c340e2c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:10.021983 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10568250-2e18-4165-b82e-2603c340e2c2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:10.023873 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 95/191] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:10.024970 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10568250-2e18-4165-b82e-2603c340e2c2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:10.140269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c3b30-1e7d-46ac-b7a5-40025e0d770c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:10.161765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:10.167077 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10568250-2e18-4165-b82e-2603c340e2c2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:10.171558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:10.187481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc83764-088f-441d-98f7-5cee8ac9c82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:10.188592 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc83764-088f-441d-98f7-5cee8ac9c82c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:10.189137 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc83764-088f-441d-98f7-5cee8ac9c82c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:10.189641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc83764-088f-441d-98f7-5cee8ac9c82c None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:10.190755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc83764-088f-441d-98f7-5cee8ac9c82c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:10.191327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc83764-088f-441d-98f7-5cee8ac9c82c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:10.193048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc83764-088f-441d-98f7-5cee8ac9c82c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:10.203886 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 97/192] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:45:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:10.209550 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc83764-088f-441d-98f7-5cee8ac9c82c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:10.216307 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b7ac6-5844-47c5-ab3a-bd3790c87307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:10.216975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b7ac6-5844-47c5-ab3a-bd3790c87307 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:10.217263 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b7ac6-5844-47c5-ab3a-bd3790c87307 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:10.217542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b7ac6-5844-47c5-ab3a-bd3790c87307 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:10.314537 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b7ac6-5844-47c5-ab3a-bd3790c87307 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:10.316400 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b7ac6-5844-47c5-ab3a-bd3790c87307 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:10.341740 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc83764-088f-441d-98f7-5cee8ac9c82c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:10.378244 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1aff2-146f-474e-bd1f-6adf09c4931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:10.379377 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1aff2-146f-474e-bd1f-6adf09c4931a None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:10.379986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1aff2-146f-474e-bd1f-6adf09c4931a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:10.380333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1aff2-146f-474e-bd1f-6adf09c4931a None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:10.381048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1aff2-146f-474e-bd1f-6adf09c4931a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:10.382681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1aff2-146f-474e-bd1f-6adf09c4931a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:10.410685 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 98/193] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:45:10 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:45:10.410685 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1aff2-146f-474e-bd1f-6adf09c4931a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:10.434072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:10.438190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 96/194] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:10.444284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b7ac6-5844-47c5-ab3a-bd3790c87307 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:10.469316 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:10.545487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1aff2-146f-474e-bd1f-6adf09c4931a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:10.595072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacc261-d4ea-41a3-b4a0-b45f1742d3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:10.596267 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacc261-d4ea-41a3-b4a0-b45f1742d3af None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:10.596802 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacc261-d4ea-41a3-b4a0-b45f1742d3af None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:10.596918 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b7ac6-5844-47c5-ab3a-bd3790c87307 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:10.597390 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacc261-d4ea-41a3-b4a0-b45f1742d3af None admin] PATH_INFO: `/v3/users/46d60b27f67a4955acb638a969d452ea` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:10.598965 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacc261-d4ea-41a3-b4a0-b45f1742d3af None admin] RBAC: Authorizing `identity:get_user(user_id=46d60b27f67a4955acb638a969d452ea)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:10.600672 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacc261-d4ea-41a3-b4a0-b45f1742d3af None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:10.602656 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 99/195] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:10 2022] GET /identity/v3/users/46d60b27f67a4955acb638a969d452ea => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:10.608970 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacc261-d4ea-41a3-b4a0-b45f1742d3af None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:10.639331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c39e6-30bf-42d1-9451-8e5c05099d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:10.640205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c39e6-30bf-42d1-9451-8e5c05099d6f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:10.640504 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c39e6-30bf-42d1-9451-8e5c05099d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:10.640763 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c39e6-30bf-42d1-9451-8e5c05099d6f None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:10.641991 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c39e6-30bf-42d1-9451-8e5c05099d6f None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:10.644436 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c39e6-30bf-42d1-9451-8e5c05099d6f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:10.645666 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 97/196] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:10 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:10.651882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c39e6-30bf-42d1-9451-8e5c05099d6f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:10.739331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:10.740940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacc261-d4ea-41a3-b4a0-b45f1742d3af None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:10.754937 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:10.794808 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c39e6-30bf-42d1-9451-8e5c05099d6f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:10.798302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d169aa7-0c8e-40b1-acb9-9334d0adf98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:10.801435 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d169aa7-0c8e-40b1-acb9-9334d0adf98c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:10.802115 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d169aa7-0c8e-40b1-acb9-9334d0adf98c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:10.802652 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d169aa7-0c8e-40b1-acb9-9334d0adf98c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:10.842911 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da20cfa-06e2-43ab-93bd-4fa04459bed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:10.843576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da20cfa-06e2-43ab-93bd-4fa04459bed7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:10.843857 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da20cfa-06e2-43ab-93bd-4fa04459bed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:10.844132 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da20cfa-06e2-43ab-93bd-4fa04459bed7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:10.844962 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da20cfa-06e2-43ab-93bd-4fa04459bed7 None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df, user.id=46d60b27f67a4955acb638a969d452ea) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:10.845274 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da20cfa-06e2-43ab-93bd-4fa04459bed7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:10.847794 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da20cfa-06e2-43ab-93bd-4fa04459bed7 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:10.857573 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 98/197] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:10 2022] GET /identity/v3/role_assignments?user.id=46d60b27f67a4955acb638a969d452ea&scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:10.893266 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d169aa7-0c8e-40b1-acb9-9334d0adf98c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:10.895346 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d169aa7-0c8e-40b1-acb9-9334d0adf98c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:11.009450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 100/198] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:11.015528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da20cfa-06e2-43ab-93bd-4fa04459bed7 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:11.178745 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da20cfa-06e2-43ab-93bd-4fa04459bed7 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:11.213877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:11.220141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63ac27-5a2f-400a-83a1-5de050312e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:11.221009 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63ac27-5a2f-400a-83a1-5de050312e48 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:11.221287 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63ac27-5a2f-400a-83a1-5de050312e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:11.221560 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63ac27-5a2f-400a-83a1-5de050312e48 None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:11.222773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63ac27-5a2f-400a-83a1-5de050312e48 None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:11.224205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63ac27-5a2f-400a-83a1-5de050312e48 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:11.225884 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 99/199] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:11 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:11.232418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d169aa7-0c8e-40b1-acb9-9334d0adf98c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:11.249322 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:11.371969 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d169aa7-0c8e-40b1-acb9-9334d0adf98c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:11.413644 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3819aad-8807-4784-9342-f46b542ddd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:11.414319 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3819aad-8807-4784-9342-f46b542ddd7b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:11.414628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3819aad-8807-4784-9342-f46b542ddd7b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:11.414916 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3819aad-8807-4784-9342-f46b542ddd7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:11.415763 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3819aad-8807-4784-9342-f46b542ddd7b None admin] RBAC: Adding query filter params (role.id=662cb9b5dff94cdb83f292ad20d1d430, scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df, user.id=46d60b27f67a4955acb638a969d452ea) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:11.416078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3819aad-8807-4784-9342-f46b542ddd7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:11.418228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3819aad-8807-4784-9342-f46b542ddd7b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:11.424185 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 101/200] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:11 2022] GET /identity/v3/role_assignments?user.id=46d60b27f67a4955acb638a969d452ea&scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df&role.id=662cb9b5dff94cdb83f292ad20d1d430 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:11.692407 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5938f5-956e-4930-b80a-ed946c1a86a6 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:11.692774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5938f5-956e-4930-b80a-ed946c1a86a6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:11.692950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5938f5-956e-4930-b80a-ed946c1a86a6 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:11.694317 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 100/201] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:11.700473 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1e1ee-bb36-44ad-8c78-944803307218 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:11.700770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1e1ee-bb36-44ad-8c78-944803307218 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:11.701055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1e1ee-bb36-44ad-8c78-944803307218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:11.724321 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1ea1e1ee-bb36-44ad-8c78-944803307218 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:11.725528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1ea1e1ee-bb36-44ad-8c78-944803307218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:11.814171 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 102/202] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:11.852092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5938f5-956e-4930-b80a-ed946c1a86a6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:12.015507 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5938f5-956e-4930-b80a-ed946c1a86a6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:12.056969 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178b054-5ea7-4918-9fb1-9cafea022518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:12.058504 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178b054-5ea7-4918-9fb1-9cafea022518 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:12.058964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178b054-5ea7-4918-9fb1-9cafea022518 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:12.059206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178b054-5ea7-4918-9fb1-9cafea022518 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:12.074355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178b054-5ea7-4918-9fb1-9cafea022518 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:12.082156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-7178b054-5ea7-4918-9fb1-9cafea022518 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:12.082156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:12.082156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:12.082156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:12.082156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:12.082156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:12.082156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:12.082156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:12.082156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:12.082156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:12.082156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:12.082156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:12.082156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:12.082156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:12.082156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:12.082156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:12.083099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:12.083099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:12.083099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:12.083099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:12.083099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:12.083099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:12.083099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:12.083099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:12.083099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:12.083099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:12.083099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:12.083099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:12.083099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:12.083099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:12.083099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:12.083099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:12.084299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:12.084299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:12.084299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:12.084299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:12.084299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:12.084299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:12.084299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:12.084299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:12.084299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:12.084299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:12.084299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:12.084299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:12.084299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 101/203] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:12.089280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1e1ee-bb36-44ad-8c78-944803307218 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:12.091461 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:12.220066 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1e1ee-bb36-44ad-8c78-944803307218 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:12.268212 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f1d78-33fe-475c-9628-0d22f7a3c279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:12.269134 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f1d78-33fe-475c-9628-0d22f7a3c279 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:12.269417 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f1d78-33fe-475c-9628-0d22f7a3c279 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:12.269688 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f1d78-33fe-475c-9628-0d22f7a3c279 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:12.270435 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f1d78-33fe-475c-9628-0d22f7a3c279 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:12.270772 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f1d78-33fe-475c-9628-0d22f7a3c279 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:12.272137 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f1d78-33fe-475c-9628-0d22f7a3c279 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:12.286183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 103/204] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:12.291743 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178b054-5ea7-4918-9fb1-9cafea022518 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:12.373150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:12.423632 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178b054-5ea7-4918-9fb1-9cafea022518 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:12.470851 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6167b-0342-4f16-b5de-436496318d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:12.471641 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6167b-0342-4f16-b5de-436496318d6c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:12.471920 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6167b-0342-4f16-b5de-436496318d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:12.472308 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6167b-0342-4f16-b5de-436496318d6c None admin] PATH_INFO: `/v3/users/81ad1712c9424afcafb9e80388158953` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:12.473511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6167b-0342-4f16-b5de-436496318d6c None admin] RBAC: Authorizing `identity:get_user(user_id=81ad1712c9424afcafb9e80388158953)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:12.474981 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6167b-0342-4f16-b5de-436496318d6c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:12.476592 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 102/205] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:12 2022] GET /identity/v3/users/81ad1712c9424afcafb9e80388158953 => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:12.482846 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f1d78-33fe-475c-9628-0d22f7a3c279 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:12.577699 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:12.631076 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f1d78-33fe-475c-9628-0d22f7a3c279 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:12.675059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b75ffa-270a-4b88-aa5e-71de8371ae83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:12.675719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b75ffa-270a-4b88-aa5e-71de8371ae83 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:12.675999 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b75ffa-270a-4b88-aa5e-71de8371ae83 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:12.676273 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b75ffa-270a-4b88-aa5e-71de8371ae83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:12.756649 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b75ffa-270a-4b88-aa5e-71de8371ae83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:12.758019 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b75ffa-270a-4b88-aa5e-71de8371ae83 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:12.843406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:12.893747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 104/206] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:12.900077 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6167b-0342-4f16-b5de-436496318d6c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:13.061446 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6167b-0342-4f16-b5de-436496318d6c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:13.097211 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:13.108726 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690b8aa-067c-47a5-b364-91f0c2b3cd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:13.109642 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690b8aa-067c-47a5-b364-91f0c2b3cd15 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:13.109923 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690b8aa-067c-47a5-b364-91f0c2b3cd15 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:13.110194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690b8aa-067c-47a5-b364-91f0c2b3cd15 None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:13.111438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690b8aa-067c-47a5-b364-91f0c2b3cd15 None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:13.112933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690b8aa-067c-47a5-b364-91f0c2b3cd15 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:13.114940 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 103/207] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:12 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:13.121062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b75ffa-270a-4b88-aa5e-71de8371ae83 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:13.266143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63f3c2-07a9-4bbd-a686-8a8dfa90d2d5 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:13.266843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63f3c2-07a9-4bbd-a686-8a8dfa90d2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:13.267333 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63f3c2-07a9-4bbd-a686-8a8dfa90d2d5 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:13.269342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 104/208] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:45:13.273390 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b75ffa-270a-4b88-aa5e-71de8371ae83 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:13.274984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbcf76-cd3b-46a4-bc6f-0cd8c6bb2703 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:13.275575 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbcf76-cd3b-46a4-bc6f-0cd8c6bb2703 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:13.276149 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbcf76-cd3b-46a4-bc6f-0cd8c6bb2703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:13.300034 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-73dbcf76-cd3b-46a4-bc6f-0cd8c6bb2703 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:13.303337 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-73dbcf76-cd3b-46a4-bc6f-0cd8c6bb2703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:13.334683 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a108c0-7e70-40d6-a577-4d812875a75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:13.335667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a108c0-7e70-40d6-a577-4d812875a75f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:13.336140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a108c0-7e70-40d6-a577-4d812875a75f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:13.336633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a108c0-7e70-40d6-a577-4d812875a75f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:13.337646 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a108c0-7e70-40d6-a577-4d812875a75f None admin] RBAC: Adding query filter params (role.id=10a2c5c4144645358318a8aee7749f84, scope.project.id=aa7a11e47f684403b0ebf54d869da5be, user.id=81ad1712c9424afcafb9e80388158953) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:13.338164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a108c0-7e70-40d6-a577-4d812875a75f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:13.342215 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a108c0-7e70-40d6-a577-4d812875a75f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:13.354322 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 105/209] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:13 2022] GET /identity/v3/role_assignments?user.id=81ad1712c9424afcafb9e80388158953&scope.project.id=aa7a11e47f684403b0ebf54d869da5be&role.id=10a2c5c4144645358318a8aee7749f84 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:13.425048 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 105/210] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:13.450280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a108c0-7e70-40d6-a577-4d812875a75f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:13.492787 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:13.588567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a108c0-7e70-40d6-a577-4d812875a75f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:13.601344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c2a50-933e-4248-9e56-fc1c21673e8d None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:13.601966 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c2a50-933e-4248-9e56-fc1c21673e8d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:13.602431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c2a50-933e-4248-9e56-fc1c21673e8d None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:13.604032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 106/211] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:45:13.609144 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e0fa9-541d-4019-a4f2-c21a8e11ad44 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:13.609676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e0fa9-541d-4019-a4f2-c21a8e11ad44 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:13.610146 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e0fa9-541d-4019-a4f2-c21a8e11ad44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:13.630143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-df0e0fa9-541d-4019-a4f2-c21a8e11ad44 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:13.631840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-df0e0fa9-541d-4019-a4f2-c21a8e11ad44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:13.641853 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e51e72-6c3d-4f25-b478-4d2aacc189aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:13.643288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e51e72-6c3d-4f25-b478-4d2aacc189aa None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:13.643573 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e51e72-6c3d-4f25-b478-4d2aacc189aa None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:13.644108 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e51e72-6c3d-4f25-b478-4d2aacc189aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:13.658305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e51e72-6c3d-4f25-b478-4d2aacc189aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:13.669545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-93e51e72-6c3d-4f25-b478-4d2aacc189aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:13.669545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:13.669545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:13.669545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:13.669545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:13.669545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:13.669545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:13.669545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:13.669545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:13.669545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:13.669545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:13.669545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:13.669545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:13.669545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:13.669545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:13.669545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:13.670495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:13.670495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:13.670495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:13.670495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:13.670495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:13.670495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:13.670495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:13.670495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:13.670495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:13.670495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:13.670495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:13.670495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:13.670495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:13.670495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:13.670495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:13.670495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:13.671633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:13.671633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:13.671633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:13.671633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:13.671633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:13.671633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:13.671633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:13.671633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:13.671633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:13.671633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:13.671633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:13.671633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:13.671633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 106/212] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:13.676850 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e51e72-6c3d-4f25-b478-4d2aacc189aa None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:13.722685 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 107/213] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:13.750390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e0fa9-541d-4019-a4f2-c21a8e11ad44 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:13.794659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:13.810509 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e51e72-6c3d-4f25-b478-4d2aacc189aa None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:13.834067 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:13.862983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274591fa-c9de-42c6-908e-64ef13a3ca1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:13.863627 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274591fa-c9de-42c6-908e-64ef13a3ca1f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:13.863870 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274591fa-c9de-42c6-908e-64ef13a3ca1f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:13.864146 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274591fa-c9de-42c6-908e-64ef13a3ca1f None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:13.864917 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274591fa-c9de-42c6-908e-64ef13a3ca1f None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:13.865224 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274591fa-c9de-42c6-908e-64ef13a3ca1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:13.867361 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274591fa-c9de-42c6-908e-64ef13a3ca1f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:13.879160 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 107/214] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:13.884908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274591fa-c9de-42c6-908e-64ef13a3ca1f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:13.904774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e0fa9-541d-4019-a4f2-c21a8e11ad44 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:13.940486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9624c-ded7-4f70-834c-60aefcb659d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:13.941325 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9624c-ded7-4f70-834c-60aefcb659d9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:13.941567 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9624c-ded7-4f70-834c-60aefcb659d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:13.941843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9624c-ded7-4f70-834c-60aefcb659d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:13.943068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9624c-ded7-4f70-834c-60aefcb659d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:13.944490 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9624c-ded7-4f70-834c-60aefcb659d9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:13.946113 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 108/215] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:13.952686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba969cee-a5de-4e6f-86fc-bed1e9d478d9 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:13.952978 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba969cee-a5de-4e6f-86fc-bed1e9d478d9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:13.953411 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba969cee-a5de-4e6f-86fc-bed1e9d478d9 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:13.954892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 109/216] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:13.955432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba969cee-a5de-4e6f-86fc-bed1e9d478d9 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:14.019768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274591fa-c9de-42c6-908e-64ef13a3ca1f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:14.057082 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d3370-8072-4487-98d1-2e058da13393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:14.057860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d3370-8072-4487-98d1-2e058da13393 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:14.058141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d3370-8072-4487-98d1-2e058da13393 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:14.058541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d3370-8072-4487-98d1-2e058da13393 None admin] PATH_INFO: `/v3/users/b03285e2aa58475a8a0e061ed20d9d26` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:14.068417 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba969cee-a5de-4e6f-86fc-bed1e9d478d9 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:14.075368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d3370-8072-4487-98d1-2e058da13393 None admin] RBAC: Authorizing `identity:get_user(user_id=b03285e2aa58475a8a0e061ed20d9d26)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:14.076822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d3370-8072-4487-98d1-2e058da13393 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:14.078552 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 108/217] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:13 2022] GET /identity/v3/users/b03285e2aa58475a8a0e061ed20d9d26 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:14.079762 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362444a-d00f-4b99-a791-690a8bff2e83 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:14.080042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362444a-d00f-4b99-a791-690a8bff2e83 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:14.080311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362444a-d00f-4b99-a791-690a8bff2e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:14.097393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f362444a-d00f-4b99-a791-690a8bff2e83 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:14.098562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f362444a-d00f-4b99-a791-690a8bff2e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:14.099063 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:14.102405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:14.112851 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3542510-98ce-45f4-adb5-350d12f8c4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:14.113579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3542510-98ce-45f4-adb5-350d12f8c4be None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:14.113810 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3542510-98ce-45f4-adb5-350d12f8c4be None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:14.114086 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3542510-98ce-45f4-adb5-350d12f8c4be None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:14.114828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3542510-98ce-45f4-adb5-350d12f8c4be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:14.116268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3542510-98ce-45f4-adb5-350d12f8c4be None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:14.140162 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 110/218] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:45:13 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:45:14.141334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3542510-98ce-45f4-adb5-350d12f8c4be None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:14.213791 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 109/219] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:14.242247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362444a-d00f-4b99-a791-690a8bff2e83 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:14.308282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3542510-98ce-45f4-adb5-350d12f8c4be None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:14.347088 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655560fc-f014-496b-8cbe-31111b0b45f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:14.347384 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655560fc-f014-496b-8cbe-31111b0b45f9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:14.347673 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655560fc-f014-496b-8cbe-31111b0b45f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:14.347953 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655560fc-f014-496b-8cbe-31111b0b45f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:14.361259 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362444a-d00f-4b99-a791-690a8bff2e83 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:14.403104 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79f01e-d316-478e-907a-6ec3242c5c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:14.404227 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79f01e-d316-478e-907a-6ec3242c5c3c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:14.404654 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79f01e-d316-478e-907a-6ec3242c5c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:14.405071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79f01e-d316-478e-907a-6ec3242c5c3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:14.408284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:14.414706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:14.418682 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79f01e-d316-478e-907a-6ec3242c5c3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:14.428061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-4d79f01e-d316-478e-907a-6ec3242c5c3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:45:14.428061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:14.428061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:14.428061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:14.428061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:14.428061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:14.428061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:14.428061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:14.428061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:14.428061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:14.428061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:14.428061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:14.428061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:14.428061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:14.428061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:14.428061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:14.428980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:14.428980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:14.428980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:14.428980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:14.428980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:14.428980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:14.428980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:14.428980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:14.428980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:14.428980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:14.428980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:14.428980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:14.428980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:14.428980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:14.428980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:14.428980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:14.429803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:14.429803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:14.429803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:14.429803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:14.429803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:14.429803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:14.429803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:14.429803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:14.429803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:14.429803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:14.429803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:45:14.429803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:14.429803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 110/220] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:45:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:45:14.434154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c951a-3ed0-41ed-8c71-40471be52c52 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:14.434154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c951a-3ed0-41ed-8c71-40471be52c52 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:14.434154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c951a-3ed0-41ed-8c71-40471be52c52 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:14.434154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 111/221] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:14.439080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c951a-3ed0-41ed-8c71-40471be52c52 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:14.440575 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655560fc-f014-496b-8cbe-31111b0b45f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:14.442412 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655560fc-f014-496b-8cbe-31111b0b45f9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:14.563805 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c951a-3ed0-41ed-8c71-40471be52c52 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:14.568959 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 111/222] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:14.574967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598dad0-2dcd-46d7-8c88-d99383bd2c2c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:14.574967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598dad0-2dcd-46d7-8c88-d99383bd2c2c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:14.574967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598dad0-2dcd-46d7-8c88-d99383bd2c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:14.589454 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-8598dad0-2dcd-46d7-8c88-d99383bd2c2c None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:14.590721 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8598dad0-2dcd-46d7-8c88-d99383bd2c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:14.615369 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5c29b-65ca-409d-b877-d443ead65df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:14.616668 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5c29b-65ca-409d-b877-d443ead65df2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:14.617576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5c29b-65ca-409d-b877-d443ead65df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:14.618098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5c29b-65ca-409d-b877-d443ead65df2 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:14.619490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5c29b-65ca-409d-b877-d443ead65df2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:14.621009 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5c29b-65ca-409d-b877-d443ead65df2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:14.623518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5c29b-65ca-409d-b877-d443ead65df2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:14.635701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 112/223] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:14.636898 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5c29b-65ca-409d-b877-d443ead65df2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:14.676198 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 112/224] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:14.678254 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598dad0-2dcd-46d7-8c88-d99383bd2c2c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:14.691347 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:14.718080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:14.772581 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5c29b-65ca-409d-b877-d443ead65df2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:14.799283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598dad0-2dcd-46d7-8c88-d99383bd2c2c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:14.817173 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c89cdb-78a7-4590-ac3e-df001c4d6a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:14.818815 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c89cdb-78a7-4590-ac3e-df001c4d6a52 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:14.819119 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c89cdb-78a7-4590-ac3e-df001c4d6a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:14.819398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c89cdb-78a7-4590-ac3e-df001c4d6a52 None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:14.823823 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c89cdb-78a7-4590-ac3e-df001c4d6a52 None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:14.825275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c89cdb-78a7-4590-ac3e-df001c4d6a52 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:14.827047 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 113/225] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:14 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:14.832785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c89cdb-78a7-4590-ac3e-df001c4d6a52 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:14.843887 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75683c47-e7d5-44a2-b1cc-b588753a92e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:14.845471 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75683c47-e7d5-44a2-b1cc-b588753a92e5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:14.845958 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75683c47-e7d5-44a2-b1cc-b588753a92e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:14.846614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75683c47-e7d5-44a2-b1cc-b588753a92e5 None admin] PATH_INFO: `/v3/users/46d60b27f67a4955acb638a969d452ea` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:14.848351 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75683c47-e7d5-44a2-b1cc-b588753a92e5 None admin] RBAC: Authorizing `identity:get_user(user_id=46d60b27f67a4955acb638a969d452ea)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:14.850069 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75683c47-e7d5-44a2-b1cc-b588753a92e5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:14.852491 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 113/226] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:14 2022] GET /identity/v3/users/46d60b27f67a4955acb638a969d452ea => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:14.853385 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75683c47-e7d5-44a2-b1cc-b588753a92e5 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:14.950440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:14.959646 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c89cdb-78a7-4590-ac3e-df001c4d6a52 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:14.990483 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:14.994785 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75683c47-e7d5-44a2-b1cc-b588753a92e5 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:15.006596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ac129-9fdd-4747-8106-aeb587cf8acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:15.006596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ac129-9fdd-4747-8106-aeb587cf8acd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:15.006596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ac129-9fdd-4747-8106-aeb587cf8acd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:15.006596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ac129-9fdd-4747-8106-aeb587cf8acd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:15.016133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ac129-9fdd-4747-8106-aeb587cf8acd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:15.024984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-871ac129-9fdd-4747-8106-aeb587cf8acd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:15.024984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:15.024984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:15.024984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:15.024984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:15.024984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:15.024984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:15.024984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:15.024984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:15.024984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:15.024984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:15.024984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:15.024984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:15.024984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:15.024984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:15.024984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:15.026675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:15.026675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:15.026675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:15.026675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:15.026675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:15.026675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:15.026675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:15.026675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:15.026675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:15.026675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:15.026675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:15.026675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:15.026675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:15.026675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:15.026675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:15.026675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:15.027883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:15.027883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:15.027883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:15.027883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:15.027883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:15.027883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:15.027883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:15.027883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:15.027883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:15.027883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:15.027883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:15.027883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:15.027883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 114/227] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:45:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:15.028767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ac129-9fdd-4747-8106-aeb587cf8acd None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:15.038488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b8249-a421-452c-965c-2643a4ad19f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:15.039303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b8249-a421-452c-965c-2643a4ad19f4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:15.039683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b8249-a421-452c-965c-2643a4ad19f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:15.040059 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b8249-a421-452c-965c-2643a4ad19f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:15.040995 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b8249-a421-452c-965c-2643a4ad19f4 None admin] RBAC: Adding query filter params (role.id=cf55e7c6398f45efaf10dec4fe153f8a, scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df, user.id=b03285e2aa58475a8a0e061ed20d9d26) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:15.041504 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b8249-a421-452c-965c-2643a4ad19f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:15.043534 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b8249-a421-452c-965c-2643a4ad19f4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:15.049914 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 114/228] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:14 2022] GET /identity/v3/role_assignments?user.id=b03285e2aa58475a8a0e061ed20d9d26&scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df&role.id=cf55e7c6398f45efaf10dec4fe153f8a => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:15.050939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b8249-a421-452c-965c-2643a4ad19f4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:15.166139 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b8249-a421-452c-965c-2643a4ad19f4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:15.198539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ac129-9fdd-4747-8106-aeb587cf8acd None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:15.211548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de15f6d-9567-41fd-8ac2-58f678a45243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:15.212150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de15f6d-9567-41fd-8ac2-58f678a45243 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:15.212427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de15f6d-9567-41fd-8ac2-58f678a45243 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:15.212700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de15f6d-9567-41fd-8ac2-58f678a45243 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:15.214037 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de15f6d-9567-41fd-8ac2-58f678a45243 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:15.214399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de15f6d-9567-41fd-8ac2-58f678a45243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:15.216167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de15f6d-9567-41fd-8ac2-58f678a45243 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:15.225950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 115/229] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:45:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:15.235726 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de15f6d-9567-41fd-8ac2-58f678a45243 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:15.239596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:15.271373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598489f-9255-4356-88c1-ab51db508732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:15.272281 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598489f-9255-4356-88c1-ab51db508732 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:15.272571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598489f-9255-4356-88c1-ab51db508732 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:15.272852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598489f-9255-4356-88c1-ab51db508732 None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:15.274193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598489f-9255-4356-88c1-ab51db508732 None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:15.276386 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598489f-9255-4356-88c1-ab51db508732 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:15.278270 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 115/230] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:15 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:15.284009 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598489f-9255-4356-88c1-ab51db508732 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:15.351762 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:15.358083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de15f6d-9567-41fd-8ac2-58f678a45243 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:15.405994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598489f-9255-4356-88c1-ab51db508732 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:15.407742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc63bc-41aa-4735-bedc-24d759958e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:15.408789 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc63bc-41aa-4735-bedc-24d759958e60 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:15.409224 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc63bc-41aa-4735-bedc-24d759958e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:15.409668 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc63bc-41aa-4735-bedc-24d759958e60 None admin] PATH_INFO: `/v3/users/46d60b27f67a4955acb638a969d452ea` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:15.412502 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc63bc-41aa-4735-bedc-24d759958e60 None admin] RBAC: Authorizing `identity:get_user(user_id=46d60b27f67a4955acb638a969d452ea)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:15.414110 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc63bc-41aa-4735-bedc-24d759958e60 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:15.415940 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 116/231] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:15 2022] GET /identity/v3/users/46d60b27f67a4955acb638a969d452ea => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:15.423808 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc63bc-41aa-4735-bedc-24d759958e60 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:15.480268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4c0e5-6b23-4fd5-bf21-00ce0bfe9ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:15.481443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4c0e5-6b23-4fd5-bf21-00ce0bfe9ad7 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:15.481875 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4c0e5-6b23-4fd5-bf21-00ce0bfe9ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:15.482296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4c0e5-6b23-4fd5-bf21-00ce0bfe9ad7 None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df/users/46d60b27f67a4955acb638a969d452ea/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:15.486372 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4c0e5-6b23-4fd5-bf21-00ce0bfe9ad7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df, user_id=46d60b27f67a4955acb638a969d452ea, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:15.490330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4c0e5-6b23-4fd5-bf21-00ce0bfe9ad7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:15.514298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 116/232] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:45:15 2022] PUT /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df/users/46d60b27f67a4955acb638a969d452ea/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:45:15.556443 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:15.601866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc63bc-41aa-4735-bedc-24d759958e60 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:15.641614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87171870-90fe-4f0e-b14e-b2cae77efa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:15.642851 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87171870-90fe-4f0e-b14e-b2cae77efa0a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:15.643282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87171870-90fe-4f0e-b14e-b2cae77efa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:15.643705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87171870-90fe-4f0e-b14e-b2cae77efa0a None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:15.645360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87171870-90fe-4f0e-b14e-b2cae77efa0a None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:15.647010 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87171870-90fe-4f0e-b14e-b2cae77efa0a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:15.648963 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 117/233] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:15 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:15.655141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4c0e5-6b23-4fd5-bf21-00ce0bfe9ad7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:15.774940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4c0e5-6b23-4fd5-bf21-00ce0bfe9ad7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:15.799049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:15.820917 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2665e-a329-409f-a21e-573c21a87831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:15.821834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2665e-a329-409f-a21e-573c21a87831 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:15.822119 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2665e-a329-409f-a21e-573c21a87831 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:15.822547 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2665e-a329-409f-a21e-573c21a87831 None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df/users/46d60b27f67a4955acb638a969d452ea/roles/662cb9b5dff94cdb83f292ad20d1d430` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:15.825231 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2665e-a329-409f-a21e-573c21a87831 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df, user_id=46d60b27f67a4955acb638a969d452ea, role_id=662cb9b5dff94cdb83f292ad20d1d430)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:15.826811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2665e-a329-409f-a21e-573c21a87831 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:15.845323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 117/234] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:45:15 2022] PUT /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df/users/46d60b27f67a4955acb638a969d452ea/roles/662cb9b5dff94cdb83f292ad20d1d430 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:45:16.862446 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8bffc-c63f-4c6b-a770-ca1188a80944 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:16.863406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8bffc-c63f-4c6b-a770-ca1188a80944 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:16.863870 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8bffc-c63f-4c6b-a770-ca1188a80944 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:16.865809 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 118/235] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:16.871756 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b68dd3-fa80-4665-be14-28612a6a4520 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:16.872628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b68dd3-fa80-4665-be14-28612a6a4520 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:16.873090 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b68dd3-fa80-4665-be14-28612a6a4520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:16.900192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-99b68dd3-fa80-4665-be14-28612a6a4520 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:16.901831 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-99b68dd3-fa80-4665-be14-28612a6a4520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:16.983566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a2960-0b20-4d08-a245-9b3c8cd327c5 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:16.983947 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a2960-0b20-4d08-a245-9b3c8cd327c5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:16.984298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a2960-0b20-4d08-a245-9b3c8cd327c5 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:16.986958 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 119/236] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:45:16.991061 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12168b76-f884-42f6-83b4-9e2bbc230eda None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:16.991433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12168b76-f884-42f6-83b4-9e2bbc230eda None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:16.991785 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12168b76-f884-42f6-83b4-9e2bbc230eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:17.005617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 118/237] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:17.010889 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-12168b76-f884-42f6-83b4-9e2bbc230eda None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:17.012095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-12168b76-f884-42f6-83b4-9e2bbc230eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:17.039864 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b68dd3-fa80-4665-be14-28612a6a4520 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:17.138913 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 120/238] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:17.178945 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12168b76-f884-42f6-83b4-9e2bbc230eda None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:17.192452 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b68dd3-fa80-4665-be14-28612a6a4520 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:17.205327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:17.256617 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:17.279451 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7326abb-765e-4ba1-a601-7949ebfc1be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:17.280284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7326abb-765e-4ba1-a601-7949ebfc1be1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:17.280570 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7326abb-765e-4ba1-a601-7949ebfc1be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:17.280850 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7326abb-765e-4ba1-a601-7949ebfc1be1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:17.295591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12168b76-f884-42f6-83b4-9e2bbc230eda None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:17.314800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7326abb-765e-4ba1-a601-7949ebfc1be1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:17.329481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-d7326abb-765e-4ba1-a601-7949ebfc1be1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:17.329481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:17.329481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:17.329481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:17.329481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:17.329481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:17.329481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:17.329481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:17.329481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:17.329481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:17.329481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:17.329481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:17.329481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:17.329481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:17.329481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:17.329481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:17.330407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:17.330407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:17.330407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:17.330407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:17.330407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:17.330407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:17.330407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:17.330407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:17.330407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:17.330407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:17.330407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:17.330407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:17.330407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:17.330407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:17.330407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:17.330407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:17.331247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:17.331247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:17.331247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:17.331247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:17.331247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:17.331247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:17.331247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:17.331247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:17.331247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:17.331247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:17.331247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:17.331247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:17.331247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 119/239] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:45:17.340300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7326abb-765e-4ba1-a601-7949ebfc1be1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:17.343123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877e561-461f-4380-bd73-ecd12e356d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:17.344151 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877e561-461f-4380-bd73-ecd12e356d6e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:17.344624 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877e561-461f-4380-bd73-ecd12e356d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:17.345090 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877e561-461f-4380-bd73-ecd12e356d6e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:17.346758 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877e561-461f-4380-bd73-ecd12e356d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:17.348406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877e561-461f-4380-bd73-ecd12e356d6e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:17.350214 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 121/240] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:17.361265 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877e561-461f-4380-bd73-ecd12e356d6e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:17.480427 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7326abb-765e-4ba1-a601-7949ebfc1be1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:17.517114 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf8f85-f1dc-4f18-8f93-8ad567a1ebff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:17.518125 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8f85-f1dc-4f18-8f93-8ad567a1ebff None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:17.518643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8f85-f1dc-4f18-8f93-8ad567a1ebff None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:17.519370 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8f85-f1dc-4f18-8f93-8ad567a1ebff None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:17.520630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf8f85-f1dc-4f18-8f93-8ad567a1ebff None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:17.521419 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf8f85-f1dc-4f18-8f93-8ad567a1ebff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:17.523038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf8f85-f1dc-4f18-8f93-8ad567a1ebff None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:17.532937 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 120/241] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:17.538438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf8f85-f1dc-4f18-8f93-8ad567a1ebff None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:17.578955 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:17.635618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877e561-461f-4380-bd73-ecd12e356d6e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:17.702001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ed619-17ad-4e79-bdfe-986f23cb3778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:17.703110 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ed619-17ad-4e79-bdfe-986f23cb3778 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:17.703612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ed619-17ad-4e79-bdfe-986f23cb3778 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:17.704074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ed619-17ad-4e79-bdfe-986f23cb3778 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:17.705094 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ed619-17ad-4e79-bdfe-986f23cb3778 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:17.706872 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ed619-17ad-4e79-bdfe-986f23cb3778 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:17.708195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf8f85-f1dc-4f18-8f93-8ad567a1ebff None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:17.716660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:17.739235 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 122/242] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:45:17 2022] POST /identity/v3/users => generated 293 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:45:17.741131 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58e896-3d7d-4b0d-a5de-0ab955827fd8 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:17.741611 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58e896-3d7d-4b0d-a5de-0ab955827fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:17.742144 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58e896-3d7d-4b0d-a5de-0ab955827fd8 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:17.744093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 123/243] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:17.752430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb2272-8c16-4943-a1af-772493ab53c0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:17.753031 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb2272-8c16-4943-a1af-772493ab53c0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:17.753524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb2272-8c16-4943-a1af-772493ab53c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:17.754142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37cc86-b378-4d5f-8dde-f8793f8ac39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:17.755143 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37cc86-b378-4d5f-8dde-f8793f8ac39a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:17.755450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37cc86-b378-4d5f-8dde-f8793f8ac39a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:17.755802 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37cc86-b378-4d5f-8dde-f8793f8ac39a None admin] PATH_INFO: `/v3/users/7a5bc3a568004256ae7a524292c8e452` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:17.772818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37cc86-b378-4d5f-8dde-f8793f8ac39a None admin] RBAC: Authorizing `identity:get_user(user_id=7a5bc3a568004256ae7a524292c8e452)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:17.774304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37cc86-b378-4d5f-8dde-f8793f8ac39a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:17.778884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 121/244] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:17 2022] GET /identity/v3/users/7a5bc3a568004256ae7a524292c8e452 => generated 297 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:17.781957 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37cc86-b378-4d5f-8dde-f8793f8ac39a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:17.793612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-eddb2272-8c16-4943-a1af-772493ab53c0 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:17.798307 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-eddb2272-8c16-4943-a1af-772493ab53c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:17.931198 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 124/245] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:17.958915 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb2272-8c16-4943-a1af-772493ab53c0 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:18.002230 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:18.029731 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37cc86-b378-4d5f-8dde-f8793f8ac39a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:18.080343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a593702-1133-4ac9-aa5e-3d5a70f06a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:18.081387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a593702-1133-4ac9-aa5e-3d5a70f06a10 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:18.081877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a593702-1133-4ac9-aa5e-3d5a70f06a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:18.082342 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a593702-1133-4ac9-aa5e-3d5a70f06a10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:18.113901 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb2272-8c16-4943-a1af-772493ab53c0 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:18.142625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:18.156039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b6a42-8e24-455d-910c-84a3aabfe728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:18.157230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b6a42-8e24-455d-910c-84a3aabfe728 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:18.157704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b6a42-8e24-455d-910c-84a3aabfe728 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:18.158280 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b6a42-8e24-455d-910c-84a3aabfe728 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:18.172081 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704b6a42-8e24-455d-910c-84a3aabfe728 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:18.173385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a593702-1133-4ac9-aa5e-3d5a70f06a10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:18.174905 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a593702-1133-4ac9-aa5e-3d5a70f06a10 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:18.181011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-704b6a42-8e24-455d-910c-84a3aabfe728 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:18.181011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:18.181011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:18.181011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:18.181011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:18.181011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:18.181011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:18.181011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:18.181011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:18.181011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:18.181011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:18.181011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:18.181011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:18.181011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:18.181011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:18.181011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:18.182433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:18.182433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:18.182433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:18.182433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:18.182433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:18.182433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:18.182433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:18.182433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:18.182433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:18.182433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:18.182433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:18.182433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:18.182433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:18.182433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:18.182433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:18.182433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:18.183394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:18.183394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:18.183394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:18.183394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:18.183394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:18.183394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:18.183394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:18.183394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:18.183394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:18.183394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:18.183394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:18.183394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:18.186370 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 125/246] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:45:18.197149 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b6a42-8e24-455d-910c-84a3aabfe728 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:18.307936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:18.333852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 122/247] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:45:18.336063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22f22a-fc0e-4a93-ac1a-14bb45da2bc2 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:18.336666 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22f22a-fc0e-4a93-ac1a-14bb45da2bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:18.337283 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22f22a-fc0e-4a93-ac1a-14bb45da2bc2 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:18.343853 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 123/248] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:18.346106 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22f22a-fc0e-4a93-ac1a-14bb45da2bc2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:18.358055 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b6a42-8e24-455d-910c-84a3aabfe728 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:18.409775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574290f-279f-4236-bd7b-20b5ee12b318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:18.410829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574290f-279f-4236-bd7b-20b5ee12b318 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:18.411697 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574290f-279f-4236-bd7b-20b5ee12b318 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:18.411990 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574290f-279f-4236-bd7b-20b5ee12b318 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:18.412990 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574290f-279f-4236-bd7b-20b5ee12b318 None admin] RBAC: Adding query filter params (name=member) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:18.413311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574290f-279f-4236-bd7b-20b5ee12b318 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:18.415345 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574290f-279f-4236-bd7b-20b5ee12b318 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:18.429607 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 126/249] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:18.431164 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c94c7-2c21-4bc0-baa8-16863b295fa0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:18.431460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c94c7-2c21-4bc0-baa8-16863b295fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:18.431831 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c94c7-2c21-4bc0-baa8-16863b295fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:18.452429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-755c94c7-2c21-4bc0-baa8-16863b295fa0 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:18.453631 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-755c94c7-2c21-4bc0-baa8-16863b295fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:18.481107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22f22a-fc0e-4a93-ac1a-14bb45da2bc2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:18.501140 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:18.517880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36294a-f94a-4b3d-ac56-0fd9103b9d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:18.519304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36294a-f94a-4b3d-ac56-0fd9103b9d30 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:18.519304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36294a-f94a-4b3d-ac56-0fd9103b9d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:18.519585 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36294a-f94a-4b3d-ac56-0fd9103b9d30 None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:18.520802 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36294a-f94a-4b3d-ac56-0fd9103b9d30 None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:18.522271 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36294a-f94a-4b3d-ac56-0fd9103b9d30 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:18.524368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 124/250] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:18 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:18.525251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36294a-f94a-4b3d-ac56-0fd9103b9d30 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:18.531687 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 127/251] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:18.533416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847f526-a992-4e57-8f99-7c9c475bafb5 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:18.533924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847f526-a992-4e57-8f99-7c9c475bafb5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:18.534409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847f526-a992-4e57-8f99-7c9c475bafb5 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:18.535891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 128/252] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:18.536835 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847f526-a992-4e57-8f99-7c9c475bafb5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:18.585412 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:18.639150 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36294a-f94a-4b3d-ac56-0fd9103b9d30 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:18.651726 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847f526-a992-4e57-8f99-7c9c475bafb5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:18.677027 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651c5ad-7900-4ff9-83b4-dce23c075d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:18.677917 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651c5ad-7900-4ff9-83b4-dce23c075d62 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:18.678246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651c5ad-7900-4ff9-83b4-dce23c075d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:18.678475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651c5ad-7900-4ff9-83b4-dce23c075d62 None admin] PATH_INFO: `/v3/users/b03285e2aa58475a8a0e061ed20d9d26` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:18.679741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651c5ad-7900-4ff9-83b4-dce23c075d62 None admin] RBAC: Authorizing `identity:get_user(user_id=b03285e2aa58475a8a0e061ed20d9d26)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:18.681286 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651c5ad-7900-4ff9-83b4-dce23c075d62 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:18.684054 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 125/253] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:18 2022] GET /identity/v3/users/b03285e2aa58475a8a0e061ed20d9d26 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:18.685268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8dfaa-9b0a-4c11-b37f-fb34af81300a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:18.685553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8dfaa-9b0a-4c11-b37f-fb34af81300a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:18.686128 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8dfaa-9b0a-4c11-b37f-fb34af81300a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:18.696245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0519c45-2530-488a-8642-6c4ba1af3c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:18.697226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0519c45-2530-488a-8642-6c4ba1af3c80 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:18.697735 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0519c45-2530-488a-8642-6c4ba1af3c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:18.698201 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0519c45-2530-488a-8642-6c4ba1af3c80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:18.699301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0519c45-2530-488a-8642-6c4ba1af3c80 None admin] RBAC: Adding query filter params (role.id=10a2c5c4144645358318a8aee7749f84, scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df, user.id=7a5bc3a568004256ae7a524292c8e452) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:18.699831 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0519c45-2530-488a-8642-6c4ba1af3c80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:18.701487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0519c45-2530-488a-8642-6c4ba1af3c80 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:18.707837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 129/254] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:18 2022] GET /identity/v3/role_assignments?user.id=7a5bc3a568004256ae7a524292c8e452&scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df&role.id=10a2c5c4144645358318a8aee7749f84 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:18.709030 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0519c45-2530-488a-8642-6c4ba1af3c80 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:18.719873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4ae8dfaa-9b0a-4c11-b37f-fb34af81300a None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:18.721104 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4ae8dfaa-9b0a-4c11-b37f-fb34af81300a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:18.758449 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:18.825710 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0519c45-2530-488a-8642-6c4ba1af3c80 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:18.863443 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0eb384-1509-4601-86e1-74a1a44a80ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:18.864611 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0eb384-1509-4601-86e1-74a1a44a80ec None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:18.865092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0eb384-1509-4601-86e1-74a1a44a80ec None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:18.865585 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0eb384-1509-4601-86e1-74a1a44a80ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:18.875138 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 126/255] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:18.875138 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8dfaa-9b0a-4c11-b37f-fb34af81300a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:18.899520 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0eb384-1509-4601-86e1-74a1a44a80ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:18.915018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-5b0eb384-1509-4601-86e1-74a1a44a80ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:45:18.915018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:18.915018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:18.915018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:18.915018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:18.915018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:18.915018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:18.915018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:18.915018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:18.915018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:18.915018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:18.915018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:18.915018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:18.915018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:18.915018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:18.915018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:18.915942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:18.915942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:18.915942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:18.915942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:18.915942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:18.915942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:18.915942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:18.915942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:18.915942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:18.915942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:18.915942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:18.915942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:18.915942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:18.915942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:18.915942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:18.915942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:18.916965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:18.916965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:18.916965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:18.916965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:18.916965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:18.916965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:18.916965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:18.916965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:18.916965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:18.916965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:18.916965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:45:18.916965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:18.916965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 130/256] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:45:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:18.917618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0eb384-1509-4601-86e1-74a1a44a80ec None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:18.939996 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:19.057821 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8dfaa-9b0a-4c11-b37f-fb34af81300a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:19.080939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:19.083867 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0eb384-1509-4601-86e1-74a1a44a80ec None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:19.120611 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b9d29-dcb8-4bf5-b037-f2468e91e1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:19.121412 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b9d29-dcb8-4bf5-b037-f2468e91e1fa None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:19.121693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b9d29-dcb8-4bf5-b037-f2468e91e1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:19.121964 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b9d29-dcb8-4bf5-b037-f2468e91e1fa None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:19.123888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b9d29-dcb8-4bf5-b037-f2468e91e1fa None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:19.126152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b9d29-dcb8-4bf5-b037-f2468e91e1fa None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:19.127907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 127/257] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:18 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:19.128826 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b9d29-dcb8-4bf5-b037-f2468e91e1fa None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:19.139233 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0c023-0856-430c-a7db-f6670959df97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:19.140063 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0c023-0856-430c-a7db-f6670959df97 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:19.140346 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0c023-0856-430c-a7db-f6670959df97 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:19.141585 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0c023-0856-430c-a7db-f6670959df97 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:19.151767 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0c023-0856-430c-a7db-f6670959df97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:19.159951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-51a0c023-0856-430c-a7db-f6670959df97 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:19.159951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:19.159951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:19.159951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:19.159951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:19.159951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:19.159951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:19.159951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:19.159951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:19.159951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:19.159951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:19.159951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:19.159951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:19.159951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:19.159951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:19.159951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:19.161056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:19.161056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:19.161056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:19.161056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:19.161056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:19.161056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:19.161056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:19.161056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:19.161056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:19.161056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:19.161056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:19.161056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:19.161056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:19.161056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:19.161056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:19.161056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:19.161957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:19.161957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:19.161957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:19.161957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:19.161957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:19.161957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:19.161957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:19.161957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:19.161957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:19.161957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:19.161957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:19.161957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:19.161957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 131/258] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:45:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:19.162674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0c023-0856-430c-a7db-f6670959df97 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:19.306449 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0c023-0856-430c-a7db-f6670959df97 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:19.314699 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b9d29-dcb8-4bf5-b037-f2468e91e1fa None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:19.337458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:19.352599 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa78cdc-6324-481e-b835-f30c2b9b8423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:19.353963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa78cdc-6324-481e-b835-f30c2b9b8423 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:19.354272 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa78cdc-6324-481e-b835-f30c2b9b8423 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:19.354550 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa78cdc-6324-481e-b835-f30c2b9b8423 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:19.355639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa78cdc-6324-481e-b835-f30c2b9b8423 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:19.355954 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa78cdc-6324-481e-b835-f30c2b9b8423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:19.357588 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa78cdc-6324-481e-b835-f30c2b9b8423 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:19.366679 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 128/259] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:19.367390 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa78cdc-6324-481e-b835-f30c2b9b8423 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:19.406599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3dea6-37c9-4f71-953f-ff6000a8b37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:19.407423 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3dea6-37c9-4f71-953f-ff6000a8b37e None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:19.407703 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3dea6-37c9-4f71-953f-ff6000a8b37e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:19.407979 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3dea6-37c9-4f71-953f-ff6000a8b37e None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df/users/b03285e2aa58475a8a0e061ed20d9d26/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:19.409990 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3dea6-37c9-4f71-953f-ff6000a8b37e None admin] RBAC: Authorizing `identity:create_grant(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df, user_id=b03285e2aa58475a8a0e061ed20d9d26, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:19.411518 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3dea6-37c9-4f71-953f-ff6000a8b37e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:19.438874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 132/260] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:45:19 2022] PUT /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df/users/b03285e2aa58475a8a0e061ed20d9d26/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:45:19.439709 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3dea6-37c9-4f71-953f-ff6000a8b37e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:19.482012 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa78cdc-6324-481e-b835-f30c2b9b8423 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:19.490385 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:19.526472 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7edf9b-6263-4593-b785-c3a254031ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:19.527360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7edf9b-6263-4593-b785-c3a254031ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:19.527649 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7edf9b-6263-4593-b785-c3a254031ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:19.527925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7edf9b-6263-4593-b785-c3a254031ad3 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:19.528689 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7edf9b-6263-4593-b785-c3a254031ad3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:19.529001 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7edf9b-6263-4593-b785-c3a254031ad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:19.531286 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7edf9b-6263-4593-b785-c3a254031ad3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:19.543213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 129/261] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:45:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:19.552600 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7edf9b-6263-4593-b785-c3a254031ad3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:19.668654 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:19.684444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3dea6-37c9-4f71-953f-ff6000a8b37e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:19.739402 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f143a0a-53ea-40b2-b509-75821e0f2a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:19.740643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f143a0a-53ea-40b2-b509-75821e0f2a60 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:19.741148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f143a0a-53ea-40b2-b509-75821e0f2a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:19.741681 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f143a0a-53ea-40b2-b509-75821e0f2a60 None admin] PATH_INFO: `/v3/users/46d60b27f67a4955acb638a969d452ea` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:19.743293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f143a0a-53ea-40b2-b509-75821e0f2a60 None admin] RBAC: Authorizing `identity:get_user(user_id=46d60b27f67a4955acb638a969d452ea)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:19.747499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f143a0a-53ea-40b2-b509-75821e0f2a60 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:19.749551 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 133/262] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:19 2022] GET /identity/v3/users/46d60b27f67a4955acb638a969d452ea => generated 283 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:19.755690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f143a0a-53ea-40b2-b509-75821e0f2a60 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:19.824953 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7edf9b-6263-4593-b785-c3a254031ad3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:19.905905 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6f939-6551-4e07-be5a-213d4df8bfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:19.906596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:19.906817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6f939-6551-4e07-be5a-213d4df8bfeb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:19.907103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6f939-6551-4e07-be5a-213d4df8bfeb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:19.907378 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6f939-6551-4e07-be5a-213d4df8bfeb None admin] PATH_INFO: `/v3/users/46d60b27f67a4955acb638a969d452ea` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:19.908579 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6f939-6551-4e07-be5a-213d4df8bfeb None admin] RBAC: Authorizing `identity:get_user(user_id=46d60b27f67a4955acb638a969d452ea)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:19.910046 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6f939-6551-4e07-be5a-213d4df8bfeb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:19.912040 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 130/263] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:19 2022] GET /identity/v3/users/46d60b27f67a4955acb638a969d452ea => generated 283 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:19.917871 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6f939-6551-4e07-be5a-213d4df8bfeb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:19.929497 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f143a0a-53ea-40b2-b509-75821e0f2a60 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:20.011288 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401ee49-de86-4c24-8ae2-2f9af1170fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:20.011958 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401ee49-de86-4c24-8ae2-2f9af1170fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:20.012243 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401ee49-de86-4c24-8ae2-2f9af1170fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:20.012521 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401ee49-de86-4c24-8ae2-2f9af1170fb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:20.085330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6f939-6551-4e07-be5a-213d4df8bfeb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:20.111197 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:20.124384 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7401ee49-de86-4c24-8ae2-2f9af1170fb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:20.125885 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7401ee49-de86-4c24-8ae2-2f9af1170fb2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:20.141034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c4f14-a738-4f39-85b0-b882855b9eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:20.141693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c4f14-a738-4f39-85b0-b882855b9eab None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:20.141973 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c4f14-a738-4f39-85b0-b882855b9eab None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:20.142247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c4f14-a738-4f39-85b0-b882855b9eab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:20.243029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:20.246700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 134/264] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:20.255456 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401ee49-de86-4c24-8ae2-2f9af1170fb2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:20.278362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c4f14-a738-4f39-85b0-b882855b9eab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:20.279869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c4f14-a738-4f39-85b0-b882855b9eab None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:20.429735 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401ee49-de86-4c24-8ae2-2f9af1170fb2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:20.489579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce913d8-b76c-4796-adc3-072f68a6b595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:20.490445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce913d8-b76c-4796-adc3-072f68a6b595 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:20.490759 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce913d8-b76c-4796-adc3-072f68a6b595 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:20.491036 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce913d8-b76c-4796-adc3-072f68a6b595 None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:20.492228 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce913d8-b76c-4796-adc3-072f68a6b595 None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:20.493765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce913d8-b76c-4796-adc3-072f68a6b595 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:20.508288 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 135/265] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:20 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:20.509410 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 131/266] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:45:20.515129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce913d8-b76c-4796-adc3-072f68a6b595 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:20.515670 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c4f14-a738-4f39-85b0-b882855b9eab None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:20.547143 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:20.637660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:20.692458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce913d8-b76c-4796-adc3-072f68a6b595 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:20.732736 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c4f14-a738-4f39-85b0-b882855b9eab None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:20.749452 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e47e4-2d5e-49ec-90b7-18b394319aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:20.750674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e47e4-2d5e-49ec-90b7-18b394319aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:20.751156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e47e4-2d5e-49ec-90b7-18b394319aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:20.751642 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e47e4-2d5e-49ec-90b7-18b394319aa3 None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:20.760955 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e47e4-2d5e-49ec-90b7-18b394319aa3 None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:20.762718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e47e4-2d5e-49ec-90b7-18b394319aa3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:20.765295 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 136/267] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:20 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:20.771323 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e47e4-2d5e-49ec-90b7-18b394319aa3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:20.820641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c0280-8221-4b96-a903-42689e03cca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:20.821376 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c0280-8221-4b96-a903-42689e03cca7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:20.821663 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c0280-8221-4b96-a903-42689e03cca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:20.821938 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c0280-8221-4b96-a903-42689e03cca7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:20.823436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c0280-8221-4b96-a903-42689e03cca7 None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df, user.id=46d60b27f67a4955acb638a969d452ea) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:20.823753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c0280-8221-4b96-a903-42689e03cca7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:20.825352 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c0280-8221-4b96-a903-42689e03cca7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:20.836351 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 132/268] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:20 2022] GET /identity/v3/role_assignments?user.id=46d60b27f67a4955acb638a969d452ea&scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 608 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:20.953391 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e47e4-2d5e-49ec-90b7-18b394319aa3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:20.995448 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:21.017306 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c47598-8284-4a64-8f57-ab10de8122be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:21.018296 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c47598-8284-4a64-8f57-ab10de8122be None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:21.019063 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c47598-8284-4a64-8f57-ab10de8122be None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:21.019610 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c47598-8284-4a64-8f57-ab10de8122be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:21.020667 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c47598-8284-4a64-8f57-ab10de8122be None admin] RBAC: Adding query filter params (role.id=662cb9b5dff94cdb83f292ad20d1d430, scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df, user.id=46d60b27f67a4955acb638a969d452ea) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:21.021181 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c47598-8284-4a64-8f57-ab10de8122be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:21.022880 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c47598-8284-4a64-8f57-ab10de8122be None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:21.031503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 137/269] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:20 2022] GET /identity/v3/role_assignments?user.id=46d60b27f67a4955acb638a969d452ea&scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df&role.id=662cb9b5dff94cdb83f292ad20d1d430 => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:21.190697 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f28ef8-310f-4d4c-88d6-88ed957ad4e7 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:21.191242 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f28ef8-310f-4d4c-88d6-88ed957ad4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:21.191242 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f28ef8-310f-4d4c-88d6-88ed957ad4e7 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:21.192982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 133/270] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:21.200230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87967a2-8733-44ea-a6f2-be1891b7c3e2 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:21.201210 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87967a2-8733-44ea-a6f2-be1891b7c3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:21.201493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87967a2-8733-44ea-a6f2-be1891b7c3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:21.227807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d87967a2-8733-44ea-a6f2-be1891b7c3e2 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:21.229753 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d87967a2-8733-44ea-a6f2-be1891b7c3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:21.321789 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 138/271] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:21.349443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f28ef8-310f-4d4c-88d6-88ed957ad4e7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:21.362646 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcb0b3-c1a0-40e7-b8d1-0d0b5b25355e None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:21.362646 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcb0b3-c1a0-40e7-b8d1-0d0b5b25355e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:21.362646 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcb0b3-c1a0-40e7-b8d1-0d0b5b25355e None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:21.363091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 139/272] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:45:21.370376 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f3514-1150-4101-bfa8-e91510d2a550 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:21.370376 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f3514-1150-4101-bfa8-e91510d2a550 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:21.370376 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f3514-1150-4101-bfa8-e91510d2a550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:21.386912 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-7f4f3514-1150-4101-bfa8-e91510d2a550 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:21.388390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7f4f3514-1150-4101-bfa8-e91510d2a550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:21.460540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:21.464703 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 140/273] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:21.488804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f28ef8-310f-4d4c-88d6-88ed957ad4e7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:21.494660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f3514-1150-4101-bfa8-e91510d2a550 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:21.540049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cca62a-e9d5-4fc8-b0ee-5eac870fa7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:21.541788 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cca62a-e9d5-4fc8-b0ee-5eac870fa7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:21.542458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cca62a-e9d5-4fc8-b0ee-5eac870fa7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:21.542964 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cca62a-e9d5-4fc8-b0ee-5eac870fa7b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:21.555881 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:21.561717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cca62a-e9d5-4fc8-b0ee-5eac870fa7b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:21.573613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-90cca62a-e9d5-4fc8-b0ee-5eac870fa7b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:21.573613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:21.573613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:21.573613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:21.573613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:21.573613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:21.573613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:21.573613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:21.573613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:21.573613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:21.573613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:21.573613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:21.573613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:21.573613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:21.573613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:21.573613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:21.574518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:21.574518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:21.574518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:21.574518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:21.574518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:21.574518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:21.574518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:21.574518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:21.574518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:21.574518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:21.574518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:21.574518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:21.574518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:21.574518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:21.574518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:21.574518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:21.576311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:21.576311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:21.576311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:21.576311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:21.576311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:21.576311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:21.576311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:21.576311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:21.576311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:21.576311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:21.576311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:21.576311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:21.579752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 134/274] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:21.592561 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cca62a-e9d5-4fc8-b0ee-5eac870fa7b6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:21.625114 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f3514-1150-4101-bfa8-e91510d2a550 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:21.668962 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94df9ab-39a8-4e3c-b3d8-1c33ad60ba2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:21.670407 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94df9ab-39a8-4e3c-b3d8-1c33ad60ba2e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:21.670923 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94df9ab-39a8-4e3c-b3d8-1c33ad60ba2e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:21.671401 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94df9ab-39a8-4e3c-b3d8-1c33ad60ba2e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:21.672811 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94df9ab-39a8-4e3c-b3d8-1c33ad60ba2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:21.674522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94df9ab-39a8-4e3c-b3d8-1c33ad60ba2e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:21.676538 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 141/275] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:21.684109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94df9ab-39a8-4e3c-b3d8-1c33ad60ba2e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:21.756992 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cca62a-e9d5-4fc8-b0ee-5eac870fa7b6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:21.810352 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574d170-ce7d-4934-81d8-c24fdf82d584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:21.811288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574d170-ce7d-4934-81d8-c24fdf82d584 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:21.811743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574d170-ce7d-4934-81d8-c24fdf82d584 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:21.812162 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574d170-ce7d-4934-81d8-c24fdf82d584 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:21.813055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574d170-ce7d-4934-81d8-c24fdf82d584 None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:21.814559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574d170-ce7d-4934-81d8-c24fdf82d584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:21.817408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574d170-ce7d-4934-81d8-c24fdf82d584 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:21.820505 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:21.829743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 135/276] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:21.835839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eb6f1-6d72-45dc-b061-d7165d2948fe None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:21.836534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eb6f1-6d72-45dc-b061-d7165d2948fe None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:21.836954 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eb6f1-6d72-45dc-b061-d7165d2948fe None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:21.839074 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 136/277] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:21.839680 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94df9ab-39a8-4e3c-b3d8-1c33ad60ba2e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:21.840238 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44eb6f1-6d72-45dc-b061-d7165d2948fe None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:21.862429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:21.889374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410080bb-63f8-4f68-8732-990a977c3777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:21.890294 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410080bb-63f8-4f68-8732-990a977c3777 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:21.890853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410080bb-63f8-4f68-8732-990a977c3777 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:21.891391 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410080bb-63f8-4f68-8732-990a977c3777 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:21.892400 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410080bb-63f8-4f68-8732-990a977c3777 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:21.894271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410080bb-63f8-4f68-8732-990a977c3777 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:21.922213 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 142/278] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:45:21 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:45:21.924624 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6dd311-0b92-4c1e-8370-8b2b48824b07 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:21.925142 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6dd311-0b92-4c1e-8370-8b2b48824b07 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:21.925892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6dd311-0b92-4c1e-8370-8b2b48824b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:21.964035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1c6dd311-0b92-4c1e-8370-8b2b48824b07 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:21.965277 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1c6dd311-0b92-4c1e-8370-8b2b48824b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:22.009350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44eb6f1-6d72-45dc-b061-d7165d2948fe None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:22.072638 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 143/279] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:22.109995 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6dd311-0b92-4c1e-8370-8b2b48824b07 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:22.111763 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc19dc-1cad-4607-b510-094a5df58d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:22.112928 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc19dc-1cad-4607-b510-094a5df58d1f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:22.113603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc19dc-1cad-4607-b510-094a5df58d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:22.114008 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc19dc-1cad-4607-b510-094a5df58d1f None admin] PATH_INFO: `/v3/users/75041b0b9e264fc9b5bf79b0f35461ef` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:22.142056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc19dc-1cad-4607-b510-094a5df58d1f None admin] RBAC: Authorizing `identity:get_user(user_id=75041b0b9e264fc9b5bf79b0f35461ef)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:22.143930 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc19dc-1cad-4607-b510-094a5df58d1f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:22.146089 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 137/280] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:21 2022] GET /identity/v3/users/75041b0b9e264fc9b5bf79b0f35461ef => generated 293 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:22.154555 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc19dc-1cad-4607-b510-094a5df58d1f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:22.239379 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:22.273787 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6dd311-0b92-4c1e-8370-8b2b48824b07 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:22.276222 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:22.297923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc19dc-1cad-4607-b510-094a5df58d1f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:22.325040 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91b140-f2f2-4c71-b711-a50fa0abc001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:22.325850 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91b140-f2f2-4c71-b711-a50fa0abc001 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:22.326772 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91b140-f2f2-4c71-b711-a50fa0abc001 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:22.326772 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91b140-f2f2-4c71-b711-a50fa0abc001 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:22.340298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91b140-f2f2-4c71-b711-a50fa0abc001 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:22.343917 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479fffc-5761-4890-97af-d3bfbb94cf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:22.344659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479fffc-5761-4890-97af-d3bfbb94cf5f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:22.344951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479fffc-5761-4890-97af-d3bfbb94cf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:22.345232 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479fffc-5761-4890-97af-d3bfbb94cf5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:22.346071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479fffc-5761-4890-97af-d3bfbb94cf5f None admin] RBAC: Adding query filter params (role.id=cf55e7c6398f45efaf10dec4fe153f8a, user.id=75041b0b9e264fc9b5bf79b0f35461ef, scope.system=all) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:22.346387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479fffc-5761-4890-97af-d3bfbb94cf5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:22.347890 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479fffc-5761-4890-97af-d3bfbb94cf5f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:22.348217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ca91b140-f2f2-4c71-b711-a50fa0abc001 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:22.348217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:22.348217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:22.348217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:22.348217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:22.348217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:22.348217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:22.348217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:22.348217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:22.348217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:22.348217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:22.348217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:22.348217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:22.348217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:22.348217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:22.348217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:22.349073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:22.349073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:22.349073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:22.349073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:22.349073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:22.349073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:22.349073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:22.349073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:22.349073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:22.349073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:22.349073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:22.349073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:22.349073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:22.349073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:22.349073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:22.349073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:22.349865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:22.349865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:22.349865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:22.349865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:22.349865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:22.349865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:22.349865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:22.349865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:22.349865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:22.349865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:22.349865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:22.349865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:22.349865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 144/281] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:45:22.351285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3efcd3-dc85-42c1-9437-93cb46212641 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:22.351449 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3efcd3-dc85-42c1-9437-93cb46212641 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:22.352577 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3efcd3-dc85-42c1-9437-93cb46212641 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:22.353134 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 145/282] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:22.359502 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80d27d-0051-4a91-87d7-3b87081169cf None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:22.360129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80d27d-0051-4a91-87d7-3b87081169cf None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:22.360299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80d27d-0051-4a91-87d7-3b87081169cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:22.364734 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 138/283] 10.222.0.70 () {56 vars in 1308 bytes} [Thu Jun 16 18:45:22 2022] GET /identity/v3/role_assignments?user.id=75041b0b9e264fc9b5bf79b0f35461ef&scope.system=all&role.id=cf55e7c6398f45efaf10dec4fe153f8a => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:22.365648 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479fffc-5761-4890-97af-d3bfbb94cf5f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:22.387691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-7b80d27d-0051-4a91-87d7-3b87081169cf None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:22.389029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7b80d27d-0051-4a91-87d7-3b87081169cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:22.514976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 146/284] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:22.541715 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80d27d-0051-4a91-87d7-3b87081169cf None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:22.556673 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479fffc-5761-4890-97af-d3bfbb94cf5f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:22.607257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2256b99-8b89-414f-9a94-070188097f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:22.608345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2256b99-8b89-414f-9a94-070188097f9d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:22.608777 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2256b99-8b89-414f-9a94-070188097f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:22.609198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2256b99-8b89-414f-9a94-070188097f9d None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:22.610092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2256b99-8b89-414f-9a94-070188097f9d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:22.610623 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2256b99-8b89-414f-9a94-070188097f9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:22.612276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2256b99-8b89-414f-9a94-070188097f9d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:22.614756 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:22.622918 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 139/285] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:22.628536 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2256b99-8b89-414f-9a94-070188097f9d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:22.743212 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2256b99-8b89-414f-9a94-070188097f9d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:22.783317 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:22.799209 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80d27d-0051-4a91-87d7-3b87081169cf None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:22.804423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c4765-d508-4408-9f4e-fdf93b4bf0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:22.806009 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c4765-d508-4408-9f4e-fdf93b4bf0fa None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:22.807604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c4765-d508-4408-9f4e-fdf93b4bf0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:22.812463 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c4765-d508-4408-9f4e-fdf93b4bf0fa None admin] PATH_INFO: `/v3/users/7a5bc3a568004256ae7a524292c8e452` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:22.813976 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c4765-d508-4408-9f4e-fdf93b4bf0fa None admin] RBAC: Authorizing `identity:get_user(user_id=7a5bc3a568004256ae7a524292c8e452)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:22.815710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c4765-d508-4408-9f4e-fdf93b4bf0fa None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:22.818307 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 140/286] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:22 2022] GET /identity/v3/users/7a5bc3a568004256ae7a524292c8e452 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:22.828469 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c4765-d508-4408-9f4e-fdf93b4bf0fa None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:22.873985 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2b001-1cb7-462a-a96b-c5f016263a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:22.875363 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2b001-1cb7-462a-a96b-c5f016263a6d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:22.877165 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2b001-1cb7-462a-a96b-c5f016263a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:22.878261 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2b001-1cb7-462a-a96b-c5f016263a6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:22.907637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2b001-1cb7-462a-a96b-c5f016263a6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:22.915279 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:22.923630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-52f2b001-1cb7-462a-a96b-c5f016263a6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:22.923630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:22.923630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:22.923630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:22.923630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:22.923630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:22.923630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:22.923630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:22.923630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:22.923630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:22.923630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:22.923630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:22.923630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:22.923630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:22.923630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:22.923630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:22.924552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:22.924552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:22.924552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:22.924552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:22.924552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:22.924552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:22.924552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:22.924552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:22.924552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:22.924552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:22.924552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:22.924552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:22.924552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:22.924552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:22.924552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:22.924552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:22.925378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:22.925378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:22.925378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:22.925378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:22.925378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:22.925378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:22.925378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:22.925378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:22.925378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:22.925378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:22.925378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:22.925378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:22.925378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 147/287] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:45:22.933217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2b001-1cb7-462a-a96b-c5f016263a6d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:23.011494 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c4765-d508-4408-9f4e-fdf93b4bf0fa None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:23.076384 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2810f-8c09-453f-964b-71d113c6d28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:23.077612 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2810f-8c09-453f-964b-71d113c6d28c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:23.078089 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2810f-8c09-453f-964b-71d113c6d28c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:23.079321 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2810f-8c09-453f-964b-71d113c6d28c None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:23.082046 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2810f-8c09-453f-964b-71d113c6d28c None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:23.083761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2810f-8c09-453f-964b-71d113c6d28c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:23.090863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 141/288] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:22 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:23.096116 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2810f-8c09-453f-964b-71d113c6d28c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:23.155020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2b001-1cb7-462a-a96b-c5f016263a6d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:23.191097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d5904-2bf6-4c38-b848-f5c6aab56d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:23.192120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d5904-2bf6-4c38-b848-f5c6aab56d16 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:23.192599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d5904-2bf6-4c38-b848-f5c6aab56d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:23.193070 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d5904-2bf6-4c38-b848-f5c6aab56d16 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:23.194029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d5904-2bf6-4c38-b848-f5c6aab56d16 None admin] RBAC: Adding query filter params (name=member) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:23.194543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d5904-2bf6-4c38-b848-f5c6aab56d16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:23.196223 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d5904-2bf6-4c38-b848-f5c6aab56d16 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:23.206014 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 148/289] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:23.211500 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d5904-2bf6-4c38-b848-f5c6aab56d16 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:23.242885 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:23.305413 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2810f-8c09-453f-964b-71d113c6d28c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:23.348116 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:23.389910 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c633b8-2552-4bdb-851d-546f129a78a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:23.390982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c633b8-2552-4bdb-851d-546f129a78a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:23.391266 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c633b8-2552-4bdb-851d-546f129a78a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:23.391543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c633b8-2552-4bdb-851d-546f129a78a4 None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df/users/7a5bc3a568004256ae7a524292c8e452/roles/10a2c5c4144645358318a8aee7749f84` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:23.395965 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c633b8-2552-4bdb-851d-546f129a78a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df, user_id=7a5bc3a568004256ae7a524292c8e452, role_id=10a2c5c4144645358318a8aee7749f84)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:23.397476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c633b8-2552-4bdb-851d-546f129a78a4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:23.419310 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d5904-2bf6-4c38-b848-f5c6aab56d16 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:23.426783 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 142/290] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:45:23 2022] PUT /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df/users/7a5bc3a568004256ae7a524292c8e452/roles/10a2c5c4144645358318a8aee7749f84 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:45:23.459421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349ace6-0ebd-4cba-a74f-de580f6bcfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:23.460340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349ace6-0ebd-4cba-a74f-de580f6bcfc1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:23.460665 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349ace6-0ebd-4cba-a74f-de580f6bcfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:23.460950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349ace6-0ebd-4cba-a74f-de580f6bcfc1 None admin] PATH_INFO: `/v3/users/b03285e2aa58475a8a0e061ed20d9d26` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:23.462311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349ace6-0ebd-4cba-a74f-de580f6bcfc1 None admin] RBAC: Authorizing `identity:get_user(user_id=b03285e2aa58475a8a0e061ed20d9d26)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:23.465190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349ace6-0ebd-4cba-a74f-de580f6bcfc1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:23.466891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 149/291] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:23 2022] GET /identity/v3/users/b03285e2aa58475a8a0e061ed20d9d26 => generated 297 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:23.472790 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c633b8-2552-4bdb-851d-546f129a78a4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:23.593713 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c633b8-2552-4bdb-851d-546f129a78a4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:23.637771 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5eab4-cc22-4b0e-a78f-4b306bdedfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:23.638461 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5eab4-cc22-4b0e-a78f-4b306bdedfb7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:23.638786 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5eab4-cc22-4b0e-a78f-4b306bdedfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:23.639069 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5eab4-cc22-4b0e-a78f-4b306bdedfb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:23.724706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:23.741218 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc5eab4-cc22-4b0e-a78f-4b306bdedfb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:23.742822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc5eab4-cc22-4b0e-a78f-4b306bdedfb7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:23.847344 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 143/292] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:23.852698 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349ace6-0ebd-4cba-a74f-de580f6bcfc1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:23.992256 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:24.012552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349ace6-0ebd-4cba-a74f-de580f6bcfc1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:24.053644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47973f-aa99-4ed1-b861-1efcd2742e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:24.054511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47973f-aa99-4ed1-b861-1efcd2742e7f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:24.054855 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47973f-aa99-4ed1-b861-1efcd2742e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:24.055128 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47973f-aa99-4ed1-b861-1efcd2742e7f None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:24.056412 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b47973f-aa99-4ed1-b861-1efcd2742e7f None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:24.058525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b47973f-aa99-4ed1-b861-1efcd2742e7f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:24.060574 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 150/293] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:23 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:24.067391 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5eab4-cc22-4b0e-a78f-4b306bdedfb7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:24.176129 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:24.198091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5eab4-cc22-4b0e-a78f-4b306bdedfb7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:24.232981 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79463904-98cd-4d70-bda6-60e4e7f1a67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:24.233650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79463904-98cd-4d70-bda6-60e4e7f1a67b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:24.233935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79463904-98cd-4d70-bda6-60e4e7f1a67b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:24.234212 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79463904-98cd-4d70-bda6-60e4e7f1a67b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:24.235071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79463904-98cd-4d70-bda6-60e4e7f1a67b None admin] RBAC: Adding query filter params (role.id=cf55e7c6398f45efaf10dec4fe153f8a, scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df, user.id=b03285e2aa58475a8a0e061ed20d9d26) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:24.235381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79463904-98cd-4d70-bda6-60e4e7f1a67b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:24.236810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79463904-98cd-4d70-bda6-60e4e7f1a67b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:24.243387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 144/294] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:24 2022] GET /identity/v3/role_assignments?user.id=b03285e2aa58475a8a0e061ed20d9d26&scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df&role.id=cf55e7c6398f45efaf10dec4fe153f8a => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:24.457129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84575e25-f399-49ba-969b-da6f941b6686 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:24.458149 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84575e25-f399-49ba-969b-da6f941b6686 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:24.458681 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84575e25-f399-49ba-969b-da6f941b6686 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:24.460611 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 151/295] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:24.466294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d962e-90af-41bd-8e84-bfd80bec23d2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:24.466832 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d962e-90af-41bd-8e84-bfd80bec23d2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:24.466907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d962e-90af-41bd-8e84-bfd80bec23d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:24.489958 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d80d962e-90af-41bd-8e84-bfd80bec23d2 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:24.491194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d80d962e-90af-41bd-8e84-bfd80bec23d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:24.579315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 145/296] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:24.609071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84575e25-f399-49ba-969b-da6f941b6686 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:24.740766 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab3742-d83e-4475-adc8-7f16aabe3628 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:24.741400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab3742-d83e-4475-adc8-7f16aabe3628 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:24.741918 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab3742-d83e-4475-adc8-7f16aabe3628 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:24.742536 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84575e25-f399-49ba-969b-da6f941b6686 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:24.743743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 146/297] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:24.749658 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c95f20-0dde-4d0c-ab0c-346efb40ccb7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:24.750192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c95f20-0dde-4d0c-ab0c-346efb40ccb7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:24.750732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c95f20-0dde-4d0c-ab0c-346efb40ccb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:24.769994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e0c95f20-0dde-4d0c-ab0c-346efb40ccb7 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:24.771561 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e0c95f20-0dde-4d0c-ab0c-346efb40ccb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:24.786843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca092c-04ca-4c92-a6ce-e7516e8f25c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:24.788035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca092c-04ca-4c92-a6ce-e7516e8f25c7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:24.788324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca092c-04ca-4c92-a6ce-e7516e8f25c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:24.788606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca092c-04ca-4c92-a6ce-e7516e8f25c7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:24.813655 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca092c-04ca-4c92-a6ce-e7516e8f25c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:24.823780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-a2ca092c-04ca-4c92-a6ce-e7516e8f25c7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:24.823780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:24.823780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:24.823780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:24.823780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:24.823780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:24.823780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:24.823780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:24.823780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:24.823780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:24.823780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:24.823780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:24.823780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:24.823780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:24.823780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:24.823780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:24.824704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:24.824704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:24.824704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:24.824704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:24.824704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:24.824704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:24.824704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:24.824704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:24.824704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:24.824704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:24.824704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:24.824704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:24.824704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:24.824704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:24.824704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:24.824704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:24.826107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:24.826107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:24.826107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:24.826107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:24.826107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:24.826107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:24.826107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:24.826107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:24.826107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:24.826107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:24.826107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:24.826107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:24.826107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 152/298] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:24.830806 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca092c-04ca-4c92-a6ce-e7516e8f25c7 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:24.845289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:24.859362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 147/299] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:24.883584 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c95f20-0dde-4d0c-ab0c-346efb40ccb7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:24.886862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:24.954609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca092c-04ca-4c92-a6ce-e7516e8f25c7 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:24.992496 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9c884-f91c-4fc7-8b0d-d36cc60bd895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:24.993281 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9c884-f91c-4fc7-8b0d-d36cc60bd895 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:24.993569 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9c884-f91c-4fc7-8b0d-d36cc60bd895 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:24.994465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9c884-f91c-4fc7-8b0d-d36cc60bd895 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:24.995358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9c884-f91c-4fc7-8b0d-d36cc60bd895 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:24.995669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9c884-f91c-4fc7-8b0d-d36cc60bd895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:24.997171 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9c884-f91c-4fc7-8b0d-d36cc60bd895 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:25.004127 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c95f20-0dde-4d0c-ab0c-346efb40ccb7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:25.007435 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 153/300] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:25.015366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9c884-f91c-4fc7-8b0d-d36cc60bd895 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:25.044192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d555e-4a6a-4d08-8ebc-13e9c58bcd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:25.045011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d555e-4a6a-4d08-8ebc-13e9c58bcd4e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:25.045295 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d555e-4a6a-4d08-8ebc-13e9c58bcd4e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:25.045571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d555e-4a6a-4d08-8ebc-13e9c58bcd4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:25.046787 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d555e-4a6a-4d08-8ebc-13e9c58bcd4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:25.048237 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d555e-4a6a-4d08-8ebc-13e9c58bcd4e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:25.049853 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 148/301] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:25.055623 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87076a79-7920-4ae0-b9a1-29aa28aca239 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:25.055914 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87076a79-7920-4ae0-b9a1-29aa28aca239 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:25.056198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87076a79-7920-4ae0-b9a1-29aa28aca239 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:25.058041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 149/302] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:25.058768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87076a79-7920-4ae0-b9a1-29aa28aca239 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:25.097582 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:25.129526 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9c884-f91c-4fc7-8b0d-d36cc60bd895 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:25.138680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:25.168640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e6c94-9bab-4bd0-9e79-7ef6b25d7fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:25.169536 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e6c94-9bab-4bd0-9e79-7ef6b25d7fcf None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:25.169847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e6c94-9bab-4bd0-9e79-7ef6b25d7fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:25.170119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e6c94-9bab-4bd0-9e79-7ef6b25d7fcf None admin] PATH_INFO: `/v3/users/f9de576b5a5c4335a51e16d834c8cb08` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:25.173027 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87076a79-7920-4ae0-b9a1-29aa28aca239 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:25.186095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e6c94-9bab-4bd0-9e79-7ef6b25d7fcf None admin] RBAC: Authorizing `identity:get_user(user_id=f9de576b5a5c4335a51e16d834c8cb08)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:25.187800 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e6c94-9bab-4bd0-9e79-7ef6b25d7fcf None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:25.194142 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 154/303] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:25 2022] GET /identity/v3/users/f9de576b5a5c4335a51e16d834c8cb08 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:25.195556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc67e3d-889b-46d1-b591-1c06c8614be6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:25.195981 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc67e3d-889b-46d1-b591-1c06c8614be6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:25.196429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc67e3d-889b-46d1-b591-1c06c8614be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:25.210645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d28798-fdff-42a7-93f2-65af61e76aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:25.211327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d28798-fdff-42a7-93f2-65af61e76aaa None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:25.211617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d28798-fdff-42a7-93f2-65af61e76aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:25.211896 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d28798-fdff-42a7-93f2-65af61e76aaa None admin] PATH_INFO: `/v3/groups` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:25.212862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d28798-fdff-42a7-93f2-65af61e76aaa None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:25.213413 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5cc67e3d-889b-46d1-b591-1c06c8614be6 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:25.214468 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d28798-fdff-42a7-93f2-65af61e76aaa None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:25.215006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5cc67e3d-889b-46d1-b591-1c06c8614be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:25.231195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 150/304] 10.222.0.70 () {60 vars in 1134 bytes} [Thu Jun 16 18:45:25 2022] POST /identity/v3/groups => generated 229 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:45:25.231195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d28798-fdff-42a7-93f2-65af61e76aaa None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:25.298438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 155/305] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:25.323343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc67e3d-889b-46d1-b591-1c06c8614be6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:25.372584 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d28798-fdff-42a7-93f2-65af61e76aaa None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:25.407143 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:25.411450 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:25.416830 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daed1d-874d-4c1b-9500-0cf530e63014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:25.417777 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daed1d-874d-4c1b-9500-0cf530e63014 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:25.418248 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daed1d-874d-4c1b-9500-0cf530e63014 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:25.418745 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daed1d-874d-4c1b-9500-0cf530e63014 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:25.419783 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daed1d-874d-4c1b-9500-0cf530e63014 None admin] RBAC: Adding query filter params (role.id=10a2c5c4144645358318a8aee7749f84, user.id=f9de576b5a5c4335a51e16d834c8cb08, scope.system=all) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:25.420284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daed1d-874d-4c1b-9500-0cf530e63014 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:25.421997 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daed1d-874d-4c1b-9500-0cf530e63014 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:25.428219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 151/306] 10.222.0.70 () {56 vars in 1308 bytes} [Thu Jun 16 18:45:25 2022] GET /identity/v3/role_assignments?user.id=f9de576b5a5c4335a51e16d834c8cb08&scope.system=all&role.id=10a2c5c4144645358318a8aee7749f84 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:25.470991 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc67e3d-889b-46d1-b591-1c06c8614be6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:25.513714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c479a-478e-485e-a59d-1196fc143c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:25.514875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c479a-478e-485e-a59d-1196fc143c39 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:25.515342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c479a-478e-485e-a59d-1196fc143c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:25.515798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c479a-478e-485e-a59d-1196fc143c39 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:25.529248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c479a-478e-485e-a59d-1196fc143c39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:25.537548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-904c479a-478e-485e-a59d-1196fc143c39 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:25.537548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:25.537548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:25.537548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:25.537548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:25.537548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:25.537548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:25.537548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:25.537548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:25.537548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:25.537548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:25.537548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:25.537548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:25.537548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:25.537548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:25.537548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:25.538494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:25.538494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:25.538494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:25.538494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:25.538494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:25.538494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:25.538494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:25.538494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:25.538494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:25.538494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:25.538494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:25.538494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:25.538494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:25.538494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:25.538494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:25.538494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:25.539420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:25.539420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:25.539420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:25.539420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:25.539420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:25.539420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:25.539420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:25.539420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:25.539420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:25.539420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:25.539420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:25.539420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:25.541615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 156/307] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:45:25.547999 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daed1d-874d-4c1b-9500-0cf530e63014 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:25.668226 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daed1d-874d-4c1b-9500-0cf530e63014 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:25.716753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5f661-b825-4aaa-9727-94a30f8f37a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:25.717430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5f661-b825-4aaa-9727-94a30f8f37a1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:25.717713 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5f661-b825-4aaa-9727-94a30f8f37a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:25.717988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5f661-b825-4aaa-9727-94a30f8f37a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:25.718812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5f661-b825-4aaa-9727-94a30f8f37a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:25.719117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5f661-b825-4aaa-9727-94a30f8f37a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:25.720569 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5f661-b825-4aaa-9727-94a30f8f37a1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:25.735546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 152/308] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:25.740723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c479a-478e-485e-a59d-1196fc143c39 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:25.854013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c479a-478e-485e-a59d-1196fc143c39 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:25.874895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:25.889922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20e2ec-d9d3-440b-a5bc-f9b3da5a53a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:25.890833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20e2ec-d9d3-440b-a5bc-f9b3da5a53a3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:25.891113 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20e2ec-d9d3-440b-a5bc-f9b3da5a53a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:25.891384 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20e2ec-d9d3-440b-a5bc-f9b3da5a53a3 None admin] PATH_INFO: `/v3/users/75041b0b9e264fc9b5bf79b0f35461ef` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:25.892579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20e2ec-d9d3-440b-a5bc-f9b3da5a53a3 None admin] RBAC: Authorizing `identity:get_user(user_id=75041b0b9e264fc9b5bf79b0f35461ef)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:25.894286 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20e2ec-d9d3-440b-a5bc-f9b3da5a53a3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:25.896231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 157/309] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:25 2022] GET /identity/v3/users/75041b0b9e264fc9b5bf79b0f35461ef => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:25.901983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5f661-b825-4aaa-9727-94a30f8f37a1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:25.974716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:26.029856 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5f661-b825-4aaa-9727-94a30f8f37a1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:26.076897 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c1486-ed10-47be-a8b4-f5a26cc7c40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:26.077799 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c1486-ed10-47be-a8b4-f5a26cc7c40d None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:26.078083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c1486-ed10-47be-a8b4-f5a26cc7c40d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:26.078363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c1486-ed10-47be-a8b4-f5a26cc7c40d None admin] PATH_INFO: `/v3/system/users/75041b0b9e264fc9b5bf79b0f35461ef/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:26.080013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c1486-ed10-47be-a8b4-f5a26cc7c40d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=75041b0b9e264fc9b5bf79b0f35461ef, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:26.081439 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c1486-ed10-47be-a8b4-f5a26cc7c40d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:26.094177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 153/310] 10.222.0.70 () {58 vars in 1334 bytes} [Thu Jun 16 18:45:25 2022] PUT /identity/v3/system/users/75041b0b9e264fc9b5bf79b0f35461ef/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:45:26.128206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d2e56-7330-448f-bb41-f1b75f2dd1fb None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:26.128819 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d2e56-7330-448f-bb41-f1b75f2dd1fb None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:26.129330 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d2e56-7330-448f-bb41-f1b75f2dd1fb None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:26.131147 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 158/311] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:26.139982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706db394-b141-4cdf-b8bc-10a103557cbb None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:26.140602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706db394-b141-4cdf-b8bc-10a103557cbb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:26.141107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706db394-b141-4cdf-b8bc-10a103557cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:26.160718 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-706db394-b141-4cdf-b8bc-10a103557cbb None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:26.162309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-706db394-b141-4cdf-b8bc-10a103557cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:26.244506 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 154/312] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:26.268463 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d2e56-7330-448f-bb41-f1b75f2dd1fb None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:26.377921 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d2e56-7330-448f-bb41-f1b75f2dd1fb None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:26.412131 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1c761-02ba-4fea-9e03-39414387f5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:26.412920 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1c761-02ba-4fea-9e03-39414387f5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:26.413253 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1c761-02ba-4fea-9e03-39414387f5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:26.413525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1c761-02ba-4fea-9e03-39414387f5d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:26.424331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c1c761-02ba-4fea-9e03-39414387f5d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:26.433015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-e8c1c761-02ba-4fea-9e03-39414387f5d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:26.433015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:26.433015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:26.433015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:26.433015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:26.433015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:26.433015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:26.433015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:26.433015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:26.433015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:26.433015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:26.433015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:26.433015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:26.433015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:26.433015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:26.433015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:26.433892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:26.433892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:26.433892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:26.433892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:26.433892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:26.433892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:26.433892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:26.433892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:26.433892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:26.433892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:26.433892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:26.433892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:26.433892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:26.433892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:26.433892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:26.433892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:26.435148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:26.435148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:26.435148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:26.435148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:26.435148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:26.435148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:26.435148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:26.435148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:26.435148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:26.435148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:26.435148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:26.435148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:26.435148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 159/313] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:26.440290 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706db394-b141-4cdf-b8bc-10a103557cbb None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:26.491652 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:26.549531 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706db394-b141-4cdf-b8bc-10a103557cbb None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:26.586069 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00d8ad-4942-4acd-8610-cfd919a6defd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:26.586775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00d8ad-4942-4acd-8610-cfd919a6defd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:26.587061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00d8ad-4942-4acd-8610-cfd919a6defd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:26.587335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00d8ad-4942-4acd-8610-cfd919a6defd None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:26.588799 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00d8ad-4942-4acd-8610-cfd919a6defd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:26.589103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00d8ad-4942-4acd-8610-cfd919a6defd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:26.590485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00d8ad-4942-4acd-8610-cfd919a6defd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:26.599216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 155/314] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:26.605582 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1c761-02ba-4fea-9e03-39414387f5d5 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:26.662984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:26.727537 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1c761-02ba-4fea-9e03-39414387f5d5 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:26.763254 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541f7f8-b19c-451d-9d8e-5ef16334b958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:26.764127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541f7f8-b19c-451d-9d8e-5ef16334b958 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:26.764408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541f7f8-b19c-451d-9d8e-5ef16334b958 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:26.764684 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541f7f8-b19c-451d-9d8e-5ef16334b958 None admin] PATH_INFO: `/v3/users/7a5bc3a568004256ae7a524292c8e452` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:26.765880 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541f7f8-b19c-451d-9d8e-5ef16334b958 None admin] RBAC: Authorizing `identity:get_user(user_id=7a5bc3a568004256ae7a524292c8e452)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:26.767344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541f7f8-b19c-451d-9d8e-5ef16334b958 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:26.769590 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 160/315] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:26 2022] GET /identity/v3/users/7a5bc3a568004256ae7a524292c8e452 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:26.774609 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00d8ad-4942-4acd-8610-cfd919a6defd None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:26.883167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00d8ad-4942-4acd-8610-cfd919a6defd None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:26.897628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:26.925269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c89ba4-e473-4f9b-a53f-cbd98b857383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:26.925931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c89ba4-e473-4f9b-a53f-cbd98b857383 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:26.926224 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c89ba4-e473-4f9b-a53f-cbd98b857383 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:26.926492 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c89ba4-e473-4f9b-a53f-cbd98b857383 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:27.013170 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c89ba4-e473-4f9b-a53f-cbd98b857383 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:27.014700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c89ba4-e473-4f9b-a53f-cbd98b857383 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:27.129041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 156/316] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:27.135461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541f7f8-b19c-451d-9d8e-5ef16334b958 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:27.250555 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541f7f8-b19c-451d-9d8e-5ef16334b958 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:27.274835 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:27.297041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01a502-d9d5-48c1-bed5-7e321f658769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:27.297919 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01a502-d9d5-48c1-bed5-7e321f658769 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:27.298203 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01a502-d9d5-48c1-bed5-7e321f658769 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:27.298480 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01a502-d9d5-48c1-bed5-7e321f658769 None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:27.299722 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01a502-d9d5-48c1-bed5-7e321f658769 None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:27.301183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01a502-d9d5-48c1-bed5-7e321f658769 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:27.303104 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 161/317] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:27 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:27.310458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c89ba4-e473-4f9b-a53f-cbd98b857383 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:27.327503 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:27.442936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c89ba4-e473-4f9b-a53f-cbd98b857383 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:27.482468 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf14cd-3c21-484e-8a2f-5e8428f6342c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:27.483171 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf14cd-3c21-484e-8a2f-5e8428f6342c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:27.483478 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf14cd-3c21-484e-8a2f-5e8428f6342c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:27.483758 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf14cd-3c21-484e-8a2f-5e8428f6342c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:27.484621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf14cd-3c21-484e-8a2f-5e8428f6342c None admin] RBAC: Adding query filter params (role.id=10a2c5c4144645358318a8aee7749f84, scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df, user.id=7a5bc3a568004256ae7a524292c8e452) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:27.484944 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf14cd-3c21-484e-8a2f-5e8428f6342c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:27.486407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf14cd-3c21-484e-8a2f-5e8428f6342c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:27.493920 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 157/318] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:27 2022] GET /identity/v3/role_assignments?user.id=7a5bc3a568004256ae7a524292c8e452&scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df&role.id=10a2c5c4144645358318a8aee7749f84 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:27.668116 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c7226-fc9d-44d8-89fa-1a783fd20bc2 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:27.668846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c7226-fc9d-44d8-89fa-1a783fd20bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:27.668846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c7226-fc9d-44d8-89fa-1a783fd20bc2 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:27.670457 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 162/319] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:27.676294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27fa48-41d2-4638-a267-1aae35052463 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:27.676821 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27fa48-41d2-4638-a267-1aae35052463 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:27.676927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27fa48-41d2-4638-a267-1aae35052463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:27.698363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7f27fa48-41d2-4638-a267-1aae35052463 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:27.699632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7f27fa48-41d2-4638-a267-1aae35052463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:27.787180 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 158/320] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:27.812075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c7226-fc9d-44d8-89fa-1a783fd20bc2 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:27.887749 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd549d-600f-42cc-b73b-85bc6e0bebd2 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:27.888086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd549d-600f-42cc-b73b-85bc6e0bebd2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:27.888364 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd549d-600f-42cc-b73b-85bc6e0bebd2 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:27.889910 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 159/321] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:27.897142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09535ade-2b64-4361-9429-8cc63b44ee2f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:27.897422 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09535ade-2b64-4361-9429-8cc63b44ee2f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:27.897707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09535ade-2b64-4361-9429-8cc63b44ee2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:27.909091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:27.918524 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-09535ade-2b64-4361-9429-8cc63b44ee2f None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:27.919733 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-09535ade-2b64-4361-9429-8cc63b44ee2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:27.934813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c7226-fc9d-44d8-89fa-1a783fd20bc2 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:27.988069 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f06412-f06d-4152-abe5-eff316032506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:27.989351 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f06412-f06d-4152-abe5-eff316032506 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:27.989818 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f06412-f06d-4152-abe5-eff316032506 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:27.990266 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f06412-f06d-4152-abe5-eff316032506 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:27.992595 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f06412-f06d-4152-abe5-eff316032506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:27.993192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 160/322] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:27.995923 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f06412-f06d-4152-abe5-eff316032506 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:28.002906 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 163/323] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:28.005591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09535ade-2b64-4361-9429-8cc63b44ee2f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:28.018332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f06412-f06d-4152-abe5-eff316032506 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:28.087525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:28.126487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09535ade-2b64-4361-9429-8cc63b44ee2f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:28.147694 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f06412-f06d-4152-abe5-eff316032506 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:28.164525 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41acdb-222c-4fea-8d45-bb9072e5631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:28.165202 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41acdb-222c-4fea-8d45-bb9072e5631b None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:28.166021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41acdb-222c-4fea-8d45-bb9072e5631b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:28.166399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41acdb-222c-4fea-8d45-bb9072e5631b None admin] PATH_INFO: `/v3/groups` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:28.167222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41acdb-222c-4fea-8d45-bb9072e5631b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:28.168680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41acdb-222c-4fea-8d45-bb9072e5631b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:28.172760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:28.195235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 161/324] 10.222.0.70 () {60 vars in 1134 bytes} [Thu Jun 16 18:45:28 2022] POST /identity/v3/groups => generated 226 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:45:28.198214 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bd089-dc84-4262-bee5-65d085ec393e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:28.199093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bd089-dc84-4262-bee5-65d085ec393e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:28.199423 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bd089-dc84-4262-bee5-65d085ec393e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:28.199704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bd089-dc84-4262-bee5-65d085ec393e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:28.212271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293bd089-dc84-4262-bee5-65d085ec393e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:28.220982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-293bd089-dc84-4262-bee5-65d085ec393e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:28.220982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:28.220982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:28.220982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:28.220982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:28.220982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:28.220982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:28.220982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:28.220982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:28.220982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:28.220982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:28.220982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:28.220982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:28.220982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:28.220982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:28.220982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:28.221998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:28.221998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:28.221998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:28.221998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:28.221998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:28.221998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:28.221998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:28.221998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:28.221998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:28.221998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:28.221998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:28.221998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:28.221998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:28.221998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:28.221998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:28.221998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:28.222879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:28.222879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:28.222879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:28.222879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:28.222879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:28.222879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:28.222879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:28.222879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:28.222879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:28.222879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:28.222879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:28.222879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:28.227004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 164/325] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:45:28.228701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41acdb-222c-4fea-8d45-bb9072e5631b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:28.357650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41acdb-222c-4fea-8d45-bb9072e5631b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:28.383380 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1ab05-3d34-4b53-83c0-3d8f484bff26 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:28.383893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1ab05-3d34-4b53-83c0-3d8f484bff26 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:28.384188 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1ab05-3d34-4b53-83c0-3d8f484bff26 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:28.386149 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 165/326] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:28.392214 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0bb06-d827-4ea5-8ca8-a95fa1d91f1d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:28.392570 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0bb06-d827-4ea5-8ca8-a95fa1d91f1d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:28.392776 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0bb06-d827-4ea5-8ca8-a95fa1d91f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:28.396553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930289eb-db7f-4a14-ad52-bca2faa19a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:28.397274 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930289eb-db7f-4a14-ad52-bca2faa19a04 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:28.397558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930289eb-db7f-4a14-ad52-bca2faa19a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:28.397925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930289eb-db7f-4a14-ad52-bca2faa19a04 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:28.399196 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930289eb-db7f-4a14-ad52-bca2faa19a04 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:28.399196 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930289eb-db7f-4a14-ad52-bca2faa19a04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:28.401169 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930289eb-db7f-4a14-ad52-bca2faa19a04 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:28.411424 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 162/327] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:28.412572 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b7a0bb06-d827-4ea5-8ca8-a95fa1d91f1d None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:28.413765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b7a0bb06-d827-4ea5-8ca8-a95fa1d91f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:28.416916 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930289eb-db7f-4a14-ad52-bca2faa19a04 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:28.489800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:28.506678 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 166/328] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:28.531907 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0bb06-d827-4ea5-8ca8-a95fa1d91f1d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:28.544694 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930289eb-db7f-4a14-ad52-bca2faa19a04 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:28.588933 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3986d6-7ffc-46f0-bd26-02ac3f4f4422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:28.589563 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:28.589952 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3986d6-7ffc-46f0-bd26-02ac3f4f4422 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:28.590292 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3986d6-7ffc-46f0-bd26-02ac3f4f4422 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:28.591029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3986d6-7ffc-46f0-bd26-02ac3f4f4422 None admin] PATH_INFO: `/v3/users/f9de576b5a5c4335a51e16d834c8cb08` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:28.592758 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3986d6-7ffc-46f0-bd26-02ac3f4f4422 None admin] RBAC: Authorizing `identity:get_user(user_id=f9de576b5a5c4335a51e16d834c8cb08)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:28.594408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3986d6-7ffc-46f0-bd26-02ac3f4f4422 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:28.596553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 163/329] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:28 2022] GET /identity/v3/users/f9de576b5a5c4335a51e16d834c8cb08 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:28.602758 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3986d6-7ffc-46f0-bd26-02ac3f4f4422 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:28.656252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0bb06-d827-4ea5-8ca8-a95fa1d91f1d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:28.698624 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa71c4-f088-4bd3-988c-896f677a0e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:28.699570 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa71c4-f088-4bd3-988c-896f677a0e66 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:28.699885 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa71c4-f088-4bd3-988c-896f677a0e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:28.700127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa71c4-f088-4bd3-988c-896f677a0e66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:28.713219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa71c4-f088-4bd3-988c-896f677a0e66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:28.722964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-dffa71c4-f088-4bd3-988c-896f677a0e66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:28.722964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:28.722964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:28.722964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:28.722964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:28.722964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:28.722964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:28.722964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:28.722964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:28.722964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:28.722964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:28.722964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:28.722964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:28.722964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:28.722964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:28.722964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:28.724425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:28.724425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:28.724425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:28.724425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:28.724425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:28.724425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:28.724425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:28.724425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:28.724425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:28.724425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:28.724425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:28.724425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:28.724425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:28.724425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:28.724425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:28.724425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:28.725755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:28.725755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:28.725755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:28.725755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:28.725755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:28.725755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:28.725755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:28.725755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:28.725755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:28.725755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:28.725755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:28.725755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:28.725755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 167/330] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:45:28.732349 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3986d6-7ffc-46f0-bd26-02ac3f4f4422 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:28.733271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa71c4-f088-4bd3-988c-896f677a0e66 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:28.763843 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:28.774002 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe7092-3d71-4c05-a2f2-4df6a0ac16fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:28.776538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe7092-3d71-4c05-a2f2-4df6a0ac16fc None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:28.776826 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe7092-3d71-4c05-a2f2-4df6a0ac16fc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:28.777153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe7092-3d71-4c05-a2f2-4df6a0ac16fc None admin] PATH_INFO: `/v3/system/users/f9de576b5a5c4335a51e16d834c8cb08/roles/10a2c5c4144645358318a8aee7749f84` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:28.778944 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe7092-3d71-4c05-a2f2-4df6a0ac16fc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f9de576b5a5c4335a51e16d834c8cb08, role_id=10a2c5c4144645358318a8aee7749f84)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:28.780480 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe7092-3d71-4c05-a2f2-4df6a0ac16fc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:28.791476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 164/331] 10.222.0.70 () {58 vars in 1334 bytes} [Thu Jun 16 18:45:28 2022] PUT /identity/v3/system/users/f9de576b5a5c4335a51e16d834c8cb08/roles/10a2c5c4144645358318a8aee7749f84 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:45:28.868930 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa71c4-f088-4bd3-988c-896f677a0e66 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:28.871165 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:28.911843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b043502-a68f-4f6e-80f0-ab665364e50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:28.912768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b043502-a68f-4f6e-80f0-ab665364e50e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:28.913195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b043502-a68f-4f6e-80f0-ab665364e50e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:28.913679 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b043502-a68f-4f6e-80f0-ab665364e50e None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:28.914707 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b043502-a68f-4f6e-80f0-ab665364e50e None admin] RBAC: Adding query filter params (name=member) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:28.915204 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b043502-a68f-4f6e-80f0-ab665364e50e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:28.917542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b043502-a68f-4f6e-80f0-ab665364e50e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:28.927579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 168/332] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:28.933594 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe7092-3d71-4c05-a2f2-4df6a0ac16fc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:29.051773 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe7092-3d71-4c05-a2f2-4df6a0ac16fc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:29.094990 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5c384-6037-453f-a19d-8b973e13fe9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:29.095944 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5c384-6037-453f-a19d-8b973e13fe9d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:29.096238 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5c384-6037-453f-a19d-8b973e13fe9d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:29.096557 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5c384-6037-453f-a19d-8b973e13fe9d None admin] PATH_INFO: `/v3/users/75041b0b9e264fc9b5bf79b0f35461ef` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:29.097840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5c384-6037-453f-a19d-8b973e13fe9d None admin] RBAC: Authorizing `identity:get_user(user_id=75041b0b9e264fc9b5bf79b0f35461ef)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:29.099322 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5c384-6037-453f-a19d-8b973e13fe9d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:29.101253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 165/333] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:28 2022] GET /identity/v3/users/75041b0b9e264fc9b5bf79b0f35461ef => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:29.106943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b043502-a68f-4f6e-80f0-ab665364e50e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:29.220705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b043502-a68f-4f6e-80f0-ab665364e50e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:29.255541 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48934f9c-749c-406f-8014-e6e7738cef3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:29.256199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48934f9c-749c-406f-8014-e6e7738cef3e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:29.256481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48934f9c-749c-406f-8014-e6e7738cef3e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:29.256759 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48934f9c-749c-406f-8014-e6e7738cef3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:29.257747 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48934f9c-749c-406f-8014-e6e7738cef3e None admin] RBAC: Adding query filter params (role.id=cf55e7c6398f45efaf10dec4fe153f8a, user.id=75041b0b9e264fc9b5bf79b0f35461ef, scope.system=all) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:29.258066 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48934f9c-749c-406f-8014-e6e7738cef3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:29.259575 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48934f9c-749c-406f-8014-e6e7738cef3e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:29.272149 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 169/334] 10.222.0.70 () {56 vars in 1308 bytes} [Thu Jun 16 18:45:29 2022] GET /identity/v3/role_assignments?user.id=75041b0b9e264fc9b5bf79b0f35461ef&scope.system=all&role.id=cf55e7c6398f45efaf10dec4fe153f8a => generated 510 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:30.641010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b114d8-75d6-4403-b0c7-08a2ea5e7601 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:30.641575 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b114d8-75d6-4403-b0c7-08a2ea5e7601 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:30.641575 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b114d8-75d6-4403-b0c7-08a2ea5e7601 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:30.643207 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 166/335] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:30.649534 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f5c54-25ca-4235-add1-be5ee9074197 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:30.650296 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f5c54-25ca-4235-add1-be5ee9074197 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:30.650616 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f5c54-25ca-4235-add1-be5ee9074197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:30.674357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-698f5c54-25ca-4235-add1-be5ee9074197 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:30.675608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-698f5c54-25ca-4235-add1-be5ee9074197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:30.698413 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:30.777253 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 170/336] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:30.812982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b114d8-75d6-4403-b0c7-08a2ea5e7601 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:30.901011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:30.964282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b114d8-75d6-4403-b0c7-08a2ea5e7601 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:31.013978 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729740e-e851-4a65-9f1c-5f9161e3aa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:31.014812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729740e-e851-4a65-9f1c-5f9161e3aa94 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:31.015091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729740e-e851-4a65-9f1c-5f9161e3aa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:31.015361 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729740e-e851-4a65-9f1c-5f9161e3aa94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:31.031512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729740e-e851-4a65-9f1c-5f9161e3aa94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:31.039276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c729740e-e851-4a65-9f1c-5f9161e3aa94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:31.039276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:31.039276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:31.039276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:31.039276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:31.039276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:31.039276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:31.039276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:31.039276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:31.039276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:31.039276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:31.039276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:31.039276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:31.039276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:31.039276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:31.039276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:31.040187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:31.040187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:31.040187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:31.040187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:31.040187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:31.040187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:31.040187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:31.040187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:31.040187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:31.040187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:31.040187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:31.040187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:31.040187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:31.040187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:31.040187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:31.040187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:31.041306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:31.041306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:31.041306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:31.041306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:31.041306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:31.041306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:31.041306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:31.041306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:31.041306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:31.041306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:31.041306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:31.041306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:31.041306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 167/337] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:31.046501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f5c54-25ca-4235-add1-be5ee9074197 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:31.067523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e2737-dcb6-4851-bce1-5a9375a375ed None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:31.067845 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e2737-dcb6-4851-bce1-5a9375a375ed None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:31.068079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e2737-dcb6-4851-bce1-5a9375a375ed None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:31.069433 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 168/338] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:31.080462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bc41a-47ca-4226-86c9-90d69de6a6e1 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:31.080773 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bc41a-47ca-4226-86c9-90d69de6a6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:31.080997 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bc41a-47ca-4226-86c9-90d69de6a6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:31.111590 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a93bc41a-47ca-4226-86c9-90d69de6a6e1 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:31.112787 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a93bc41a-47ca-4226-86c9-90d69de6a6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:31.178651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f5c54-25ca-4235-add1-be5ee9074197 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:31.219345 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7455a1c-d667-4175-9d35-82ab4a503df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:31.220019 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7455a1c-d667-4175-9d35-82ab4a503df4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:31.220298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7455a1c-d667-4175-9d35-82ab4a503df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:31.220595 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7455a1c-d667-4175-9d35-82ab4a503df4 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:31.221340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7455a1c-d667-4175-9d35-82ab4a503df4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:31.221643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7455a1c-d667-4175-9d35-82ab4a503df4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:31.222539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 169/339] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:31.223055 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7455a1c-d667-4175-9d35-82ab4a503df4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:31.242107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 171/340] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:31.247854 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bc41a-47ca-4226-86c9-90d69de6a6e1 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:31.274650 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7455a1c-d667-4175-9d35-82ab4a503df4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:31.305356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:31.319793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:31.428928 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bc41a-47ca-4226-86c9-90d69de6a6e1 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:31.441689 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7455a1c-d667-4175-9d35-82ab4a503df4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:31.474476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35ea17-f829-4a5a-8bc3-19dbec493a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:31.475540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35ea17-f829-4a5a-8bc3-19dbec493a10 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:31.476025 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35ea17-f829-4a5a-8bc3-19dbec493a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:31.476566 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35ea17-f829-4a5a-8bc3-19dbec493a10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:31.511594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a616c-c59c-4b9d-9353-814953eafd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:31.514202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a616c-c59c-4b9d-9353-814953eafd3c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:31.516045 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a616c-c59c-4b9d-9353-814953eafd3c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:31.516553 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a616c-c59c-4b9d-9353-814953eafd3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:31.545993 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a616c-c59c-4b9d-9353-814953eafd3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:31.554312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-4d3a616c-c59c-4b9d-9353-814953eafd3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:31.554312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:31.554312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:31.554312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:31.554312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:31.554312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:31.554312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:31.554312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:31.554312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:31.554312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:31.554312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:31.554312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:31.554312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:31.554312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:31.554312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:31.554312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:31.555392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:31.555392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:31.555392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:31.555392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:31.555392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:31.555392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:31.555392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:31.555392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:31.555392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:31.555392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:31.555392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:31.555392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:31.555392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:31.555392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:31.555392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:31.555392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:31.556230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:31.556230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:31.556230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:31.556230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:31.556230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:31.556230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:31.556230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:31.556230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:31.556230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:31.556230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:31.556230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:31.556230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:31.563768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 172/341] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:45:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:45:31.569463 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a616c-c59c-4b9d-9353-814953eafd3c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:31.594490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a35ea17-f829-4a5a-8bc3-19dbec493a10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:31.596387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a35ea17-f829-4a5a-8bc3-19dbec493a10 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:31.655713 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:31.717724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:31.752269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 170/342] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:31.756956 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a616c-c59c-4b9d-9353-814953eafd3c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:31.758107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35ea17-f829-4a5a-8bc3-19dbec493a10 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:31.818599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98f59b-bd3b-494a-80a5-16ee9dbcc1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:31.818599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98f59b-bd3b-494a-80a5-16ee9dbcc1ac None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:31.818599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98f59b-bd3b-494a-80a5-16ee9dbcc1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:31.818599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98f59b-bd3b-494a-80a5-16ee9dbcc1ac None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:31.819021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98f59b-bd3b-494a-80a5-16ee9dbcc1ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:31.819021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98f59b-bd3b-494a-80a5-16ee9dbcc1ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:31.819021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98f59b-bd3b-494a-80a5-16ee9dbcc1ac None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:31.828971 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 173/343] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:45:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:31.839617 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98f59b-bd3b-494a-80a5-16ee9dbcc1ac None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:31.901013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35ea17-f829-4a5a-8bc3-19dbec493a10 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:31.957932 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3135cd-4131-46a9-9c57-b857de3024bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:31.963312 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3135cd-4131-46a9-9c57-b857de3024bd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:31.963867 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3135cd-4131-46a9-9c57-b857de3024bd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:31.964349 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3135cd-4131-46a9-9c57-b857de3024bd None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:31.965853 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3135cd-4131-46a9-9c57-b857de3024bd None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:31.967534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3135cd-4131-46a9-9c57-b857de3024bd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:31.969509 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 171/344] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:31 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:31.974987 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3135cd-4131-46a9-9c57-b857de3024bd None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:32.010015 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:32.045109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98f59b-bd3b-494a-80a5-16ee9dbcc1ac None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:32.105079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ccbdd-83f0-4b6a-ba17-6c0291a00c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:32.105739 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ccbdd-83f0-4b6a-ba17-6c0291a00c36 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:32.106067 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ccbdd-83f0-4b6a-ba17-6c0291a00c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:32.106347 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ccbdd-83f0-4b6a-ba17-6c0291a00c36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:32.115641 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:32.170453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3135cd-4131-46a9-9c57-b857de3024bd None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:32.227156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa2d640-c42e-4faf-994f-595774f8e7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:32.227669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ccbdd-83f0-4b6a-ba17-6c0291a00c36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:32.228882 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2d640-c42e-4faf-994f-595774f8e7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:32.229377 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2d640-c42e-4faf-994f-595774f8e7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:32.229478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ccbdd-83f0-4b6a-ba17-6c0291a00c36 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:32.229948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2d640-c42e-4faf-994f-595774f8e7c5 None admin] PATH_INFO: `/v3/groups/859c364715ee47fabc3a3027a878e3d9` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:32.240979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa2d640-c42e-4faf-994f-595774f8e7c5 None admin] RBAC: Authorizing `identity:get_group(group_id=859c364715ee47fabc3a3027a878e3d9)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:32.242800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa2d640-c42e-4faf-994f-595774f8e7c5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:32.245261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 172/345] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Jun 16 18:45:31 2022] GET /identity/v3/groups/859c364715ee47fabc3a3027a878e3d9 => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:32.251341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa2d640-c42e-4faf-994f-595774f8e7c5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:32.363438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:32.391104 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa2d640-c42e-4faf-994f-595774f8e7c5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:32.411030 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 174/346] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:32.412453 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504cdac-0233-478c-961c-059b847a52c4 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:32.412742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504cdac-0233-478c-961c-059b847a52c4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:32.413020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504cdac-0233-478c-961c-059b847a52c4 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:32.414355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 175/347] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:32.415641 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605094b-521f-44b9-b026-f6ab33ef4706 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:32.415927 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605094b-521f-44b9-b026-f6ab33ef4706 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:32.416202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605094b-521f-44b9-b026-f6ab33ef4706 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:32.417516 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 176/348] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:32.418248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605094b-521f-44b9-b026-f6ab33ef4706 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:32.453208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd002c08-c023-4560-84fd-1d559be1f76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:32.453880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd002c08-c023-4560-84fd-1d559be1f76d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:32.454163 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd002c08-c023-4560-84fd-1d559be1f76d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:32.454439 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd002c08-c023-4560-84fd-1d559be1f76d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:32.455296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd002c08-c023-4560-84fd-1d559be1f76d None admin] RBAC: Adding query filter params (group.id=859c364715ee47fabc3a3027a878e3d9, role.id=cf55e7c6398f45efaf10dec4fe153f8a, scope.project.id=aa7a11e47f684403b0ebf54d869da5be) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:32.458072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd002c08-c023-4560-84fd-1d559be1f76d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:32.458072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd002c08-c023-4560-84fd-1d559be1f76d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:32.464034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 173/349] 10.222.0.70 () {56 vars in 1376 bytes} [Thu Jun 16 18:45:32 2022] GET /identity/v3/role_assignments?group.id=859c364715ee47fabc3a3027a878e3d9&scope.project.id=aa7a11e47f684403b0ebf54d869da5be&role.id=cf55e7c6398f45efaf10dec4fe153f8a => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:32.465228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9080aaf-9d5c-48aa-8fb0-ce629bdbe7aa None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:32.465622 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9080aaf-9d5c-48aa-8fb0-ce629bdbe7aa None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:32.465793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9080aaf-9d5c-48aa-8fb0-ce629bdbe7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:32.486904 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:32.488637 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b9080aaf-9d5c-48aa-8fb0-ce629bdbe7aa None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:32.489313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b9080aaf-9d5c-48aa-8fb0-ce629bdbe7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:32.582431 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 174/350] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:32.584170 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7940a7c-4961-42c0-8eda-93f70ecb2c7f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:32.584455 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605094b-521f-44b9-b026-f6ab33ef4706 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:32.584784 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7940a7c-4961-42c0-8eda-93f70ecb2c7f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:32.585208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7940a7c-4961-42c0-8eda-93f70ecb2c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:32.610714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a7940a7c-4961-42c0-8eda-93f70ecb2c7f None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:32.612982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a7940a7c-4961-42c0-8eda-93f70ecb2c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:32.631101 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53774edb-7fbd-4144-9c98-5a8fc101dce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:32.632140 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53774edb-7fbd-4144-9c98-5a8fc101dce8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:32.632544 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53774edb-7fbd-4144-9c98-5a8fc101dce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:32.632919 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53774edb-7fbd-4144-9c98-5a8fc101dce8 None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:32.635035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53774edb-7fbd-4144-9c98-5a8fc101dce8 None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:32.636613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53774edb-7fbd-4144-9c98-5a8fc101dce8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:32.638535 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 177/351] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:32 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:32.639409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53774edb-7fbd-4144-9c98-5a8fc101dce8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:32.710936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 175/352] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:32.713284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7940a7c-4961-42c0-8eda-93f70ecb2c7f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:32.717178 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:32.781606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53774edb-7fbd-4144-9c98-5a8fc101dce8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:32.785798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:32.835097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e878404-4d7c-44a8-84c0-54adce0f4a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:32.836285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e878404-4d7c-44a8-84c0-54adce0f4a7b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:32.836613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7940a7c-4961-42c0-8eda-93f70ecb2c7f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:32.837951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e878404-4d7c-44a8-84c0-54adce0f4a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:32.838451 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e878404-4d7c-44a8-84c0-54adce0f4a7b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:32.851074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e878404-4d7c-44a8-84c0-54adce0f4a7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:32.860600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-3e878404-4d7c-44a8-84c0-54adce0f4a7b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:32.860600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:32.860600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:32.860600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:32.860600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:32.860600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:32.860600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:32.860600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:32.860600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:32.860600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:32.860600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:32.860600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:32.860600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:32.860600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:32.860600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:32.860600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:32.861557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:32.861557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:32.861557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:32.861557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:32.861557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:32.861557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:32.861557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:32.861557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:32.861557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:32.861557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:32.861557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:32.861557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:32.861557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:32.861557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:32.861557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:32.861557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:32.862386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:32.862386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:32.862386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:32.862386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:32.862386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:32.862386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:32.862386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:32.862386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:32.862386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:32.862386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:32.862386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:32.862386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:32.864853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 178/353] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:32.865841 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e878404-4d7c-44a8-84c0-54adce0f4a7b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:32.874338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66530371-eae7-4cc1-89c5-36c53855ddfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:32.875239 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66530371-eae7-4cc1-89c5-36c53855ddfa None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:32.875521 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66530371-eae7-4cc1-89c5-36c53855ddfa None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:32.875793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66530371-eae7-4cc1-89c5-36c53855ddfa None admin] PATH_INFO: `/v3/groups/859c364715ee47fabc3a3027a878e3d9` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:32.877170 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66530371-eae7-4cc1-89c5-36c53855ddfa None admin] RBAC: Authorizing `identity:get_group(group_id=859c364715ee47fabc3a3027a878e3d9)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:32.878515 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66530371-eae7-4cc1-89c5-36c53855ddfa None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:32.880574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 176/354] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Jun 16 18:45:32 2022] GET /identity/v3/groups/859c364715ee47fabc3a3027a878e3d9 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:32.881458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66530371-eae7-4cc1-89c5-36c53855ddfa None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:32.973664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:32.978453 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e878404-4d7c-44a8-84c0-54adce0f4a7b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:33.009756 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66530371-eae7-4cc1-89c5-36c53855ddfa None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:33.015379 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7013867-f8c2-42d7-86a3-a1c80071fa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:33.016514 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7013867-f8c2-42d7-86a3-a1c80071fa45 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:33.017047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7013867-f8c2-42d7-86a3-a1c80071fa45 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:33.017715 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7013867-f8c2-42d7-86a3-a1c80071fa45 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:33.030034 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7013867-f8c2-42d7-86a3-a1c80071fa45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:33.038329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-f7013867-f8c2-42d7-86a3-a1c80071fa45 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:33.038329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:33.038329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:33.038329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:33.038329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:33.038329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:33.038329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:33.038329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:33.038329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:33.038329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:33.038329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:33.038329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:33.038329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:33.038329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:33.038329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:33.038329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:33.039245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:33.039245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:33.039245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:33.039245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:33.039245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:33.039245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:33.039245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:33.039245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:33.039245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:33.039245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:33.039245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:33.039245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:33.039245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:33.039245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:33.039245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:33.039245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:33.040051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:33.040051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:33.040051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:33.040051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:33.040051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:33.040051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:33.040051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:33.040051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:33.040051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:33.040051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:33.040051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:45:33.040051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:33.042791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 179/355] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:33.043787 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7013867-f8c2-42d7-86a3-a1c80071fa45 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:33.052323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b99c2-94cc-4e3e-abbc-d14ac773210b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:33.053008 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b99c2-94cc-4e3e-abbc-d14ac773210b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:33.053291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b99c2-94cc-4e3e-abbc-d14ac773210b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:33.053565 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b99c2-94cc-4e3e-abbc-d14ac773210b None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:33.054317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b99c2-94cc-4e3e-abbc-d14ac773210b None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:33.056819 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b99c2-94cc-4e3e-abbc-d14ac773210b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:33.057608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b99c2-94cc-4e3e-abbc-d14ac773210b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:33.067709 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 177/356] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:33.068310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b99c2-94cc-4e3e-abbc-d14ac773210b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:33.083501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:33.176033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7013867-f8c2-42d7-86a3-a1c80071fa45 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:33.184247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b99c2-94cc-4e3e-abbc-d14ac773210b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:33.214794 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fc928-e50c-41dd-a49f-8958669cbc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:33.215815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fc928-e50c-41dd-a49f-8958669cbc51 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:33.216297 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fc928-e50c-41dd-a49f-8958669cbc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:33.216771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fc928-e50c-41dd-a49f-8958669cbc51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:33.218149 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fc928-e50c-41dd-a49f-8958669cbc51 None admin] RBAC: Adding query filter params (group.id=859c364715ee47fabc3a3027a878e3d9, role.id=662cb9b5dff94cdb83f292ad20d1d430, scope.project.id=aa7a11e47f684403b0ebf54d869da5be) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:33.218755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fc928-e50c-41dd-a49f-8958669cbc51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:33.220782 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fc928-e50c-41dd-a49f-8958669cbc51 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:33.222326 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076dbf5e-f3e5-4a94-9924-58174c5a43b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:33.223033 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dbf5e-f3e5-4a94-9924-58174c5a43b2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:33.223316 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dbf5e-f3e5-4a94-9924-58174c5a43b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:33.223593 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dbf5e-f3e5-4a94-9924-58174c5a43b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:33.224458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076dbf5e-f3e5-4a94-9924-58174c5a43b2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:33.224763 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076dbf5e-f3e5-4a94-9924-58174c5a43b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:33.226270 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076dbf5e-f3e5-4a94-9924-58174c5a43b2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:33.227820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:33.229217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 180/357] 10.222.0.70 () {56 vars in 1376 bytes} [Thu Jun 16 18:45:33 2022] GET /identity/v3/role_assignments?group.id=859c364715ee47fabc3a3027a878e3d9&scope.project.id=aa7a11e47f684403b0ebf54d869da5be&role.id=662cb9b5dff94cdb83f292ad20d1d430 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:33.230287 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fc928-e50c-41dd-a49f-8958669cbc51 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:33.237833 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 178/358] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:33.244275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076dbf5e-f3e5-4a94-9924-58174c5a43b2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:33.327744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:33.340620 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fc928-e50c-41dd-a49f-8958669cbc51 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:33.366474 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076dbf5e-f3e5-4a94-9924-58174c5a43b2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:33.379080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733bff2-7c7b-4981-837f-a1bc3269911f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:33.379764 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733bff2-7c7b-4981-837f-a1bc3269911f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:33.380049 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733bff2-7c7b-4981-837f-a1bc3269911f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:33.380339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733bff2-7c7b-4981-837f-a1bc3269911f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:33.408705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c307b79-4611-4421-b805-ff1d246d3783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:33.409962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c307b79-4611-4421-b805-ff1d246d3783 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:33.410542 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c307b79-4611-4421-b805-ff1d246d3783 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:33.411054 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c307b79-4611-4421-b805-ff1d246d3783 None admin] PATH_INFO: `/v3/users/f9de576b5a5c4335a51e16d834c8cb08` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:33.412812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c307b79-4611-4421-b805-ff1d246d3783 None admin] RBAC: Authorizing `identity:get_user(user_id=f9de576b5a5c4335a51e16d834c8cb08)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:33.414493 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c307b79-4611-4421-b805-ff1d246d3783 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:33.418110 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 179/359] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:45:33 2022] GET /identity/v3/users/f9de576b5a5c4335a51e16d834c8cb08 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:33.423655 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c307b79-4611-4421-b805-ff1d246d3783 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:33.458112 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733bff2-7c7b-4981-837f-a1bc3269911f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:33.459850 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733bff2-7c7b-4981-837f-a1bc3269911f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:33.505839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:33.566943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:33.577114 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 181/360] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:33.582771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733bff2-7c7b-4981-837f-a1bc3269911f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:33.613506 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c307b79-4611-4421-b805-ff1d246d3783 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:33.678320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f571224-0471-4578-bce3-89d9623b1420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:33.679477 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f571224-0471-4578-bce3-89d9623b1420 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:33.680023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f571224-0471-4578-bce3-89d9623b1420 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:33.680534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f571224-0471-4578-bce3-89d9623b1420 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:33.682389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f571224-0471-4578-bce3-89d9623b1420 None admin] RBAC: Adding query filter params (role.id=10a2c5c4144645358318a8aee7749f84, user.id=f9de576b5a5c4335a51e16d834c8cb08, scope.system=all) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:33.683154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f571224-0471-4578-bce3-89d9623b1420 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:33.690598 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f571224-0471-4578-bce3-89d9623b1420 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:33.690813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733bff2-7c7b-4981-837f-a1bc3269911f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:33.711084 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 180/361] 10.222.0.70 () {56 vars in 1308 bytes} [Thu Jun 16 18:45:33 2022] GET /identity/v3/role_assignments?user.id=f9de576b5a5c4335a51e16d834c8cb08&scope.system=all&role.id=10a2c5c4144645358318a8aee7749f84 => generated 510 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:33.730922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651823a-fdc7-429d-a5d0-852066677890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:33.732795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651823a-fdc7-429d-a5d0-852066677890 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:33.733077 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651823a-fdc7-429d-a5d0-852066677890 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:33.733357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651823a-fdc7-429d-a5d0-852066677890 None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:33.734757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651823a-fdc7-429d-a5d0-852066677890 None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:33.736253 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651823a-fdc7-429d-a5d0-852066677890 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:33.738409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 182/362] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:33 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:33.744298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f571224-0471-4578-bce3-89d9623b1420 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:33.818841 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d0c01-66d5-4068-b157-4fc1534ba998 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:33.819359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d0c01-66d5-4068-b157-4fc1534ba998 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:33.819434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d0c01-66d5-4068-b157-4fc1534ba998 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:33.820989 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 183/363] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:45:33.826637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5592a-f121-4d2c-a31d-eaf25797785d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:33.826929 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5592a-f121-4d2c-a31d-eaf25797785d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:33.827210 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5592a-f121-4d2c-a31d-eaf25797785d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:33.846604 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e7d5592a-f121-4d2c-a31d-eaf25797785d None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:33.847820 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e7d5592a-f121-4d2c-a31d-eaf25797785d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:33.874921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f571224-0471-4578-bce3-89d9623b1420 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:33.906923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:33.916547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:33.933615 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3bc14-9ea9-4e81-b56b-260b3e5cea9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:33.934502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3bc14-9ea9-4e81-b56b-260b3e5cea9b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:33.934814 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3bc14-9ea9-4e81-b56b-260b3e5cea9b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:33.935141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3bc14-9ea9-4e81-b56b-260b3e5cea9b None admin] PATH_INFO: `/v3/groups/859c364715ee47fabc3a3027a878e3d9` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:33.936152 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 184/364] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:33.936789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3bc14-9ea9-4e81-b56b-260b3e5cea9b None admin] RBAC: Authorizing `identity:get_group(group_id=859c364715ee47fabc3a3027a878e3d9)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:33.938352 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3bc14-9ea9-4e81-b56b-260b3e5cea9b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:33.940060 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 181/365] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Jun 16 18:45:33 2022] GET /identity/v3/groups/859c364715ee47fabc3a3027a878e3d9 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:33.951226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5592a-f121-4d2c-a31d-eaf25797785d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:34.032197 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3bc14-9ea9-4e81-b56b-260b3e5cea9b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:34.086816 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5592a-f121-4d2c-a31d-eaf25797785d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:34.128985 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d2895-7be9-4c2e-8b8d-e5142677fef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:34.129672 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d2895-7be9-4c2e-8b8d-e5142677fef9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:34.129958 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d2895-7be9-4c2e-8b8d-e5142677fef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:34.130328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d2895-7be9-4c2e-8b8d-e5142677fef9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:34.131192 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d2895-7be9-4c2e-8b8d-e5142677fef9 None admin] RBAC: Adding query filter params (group.id=859c364715ee47fabc3a3027a878e3d9, role.id=cf55e7c6398f45efaf10dec4fe153f8a, scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:34.131506 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d2895-7be9-4c2e-8b8d-e5142677fef9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:34.132940 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d2895-7be9-4c2e-8b8d-e5142677fef9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:34.140680 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 185/366] 10.222.0.70 () {56 vars in 1376 bytes} [Thu Jun 16 18:45:33 2022] GET /identity/v3/role_assignments?group.id=859c364715ee47fabc3a3027a878e3d9&scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df&role.id=cf55e7c6398f45efaf10dec4fe153f8a => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:34.161956 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a677f95-9ef0-415d-86ab-c9adaaaa1d85 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:34.162543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a677f95-9ef0-415d-86ab-c9adaaaa1d85 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:34.162847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a677f95-9ef0-415d-86ab-c9adaaaa1d85 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:34.164492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 186/367] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:34.191935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89324e-18a3-4347-b572-3bdf4028e696 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:34.192282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89324e-18a3-4347-b572-3bdf4028e696 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:34.192509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89324e-18a3-4347-b572-3bdf4028e696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:34.205209 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3bc14-9ea9-4e81-b56b-260b3e5cea9b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:34.218715 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2b89324e-18a3-4347-b572-3bdf4028e696 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:34.220018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2b89324e-18a3-4347-b572-3bdf4028e696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:34.257615 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccddfe58-65fa-4905-9f00-999a68f1d099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:34.258455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccddfe58-65fa-4905-9f00-999a68f1d099 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:34.258774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccddfe58-65fa-4905-9f00-999a68f1d099 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:34.262059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccddfe58-65fa-4905-9f00-999a68f1d099 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:34.293158 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccddfe58-65fa-4905-9f00-999a68f1d099 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:34.308278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ccddfe58-65fa-4905-9f00-999a68f1d099 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:34.308278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:34.308278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:34.308278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:34.308278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:34.308278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:34.308278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:34.308278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:34.308278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:34.308278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:34.308278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:34.308278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:34.308278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:34.308278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:34.308278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:34.308278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:34.309254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:34.309254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:34.309254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:34.309254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:34.309254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:34.309254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:34.309254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:34.309254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:34.309254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:34.309254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:34.309254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:34.309254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:34.309254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:34.309254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:34.309254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:34.309254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:34.310092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:34.310092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:34.310092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:34.310092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:34.310092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:34.310092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:34.310092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:34.310092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:34.310092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:34.310092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:34.310092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:34.310092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:34.312057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 182/368] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:45:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:34.322760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:34.323393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccddfe58-65fa-4905-9f00-999a68f1d099 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:34.366317 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 187/369] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:34.394555 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:34.403000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89324e-18a3-4347-b572-3bdf4028e696 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:34.497272 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccddfe58-65fa-4905-9f00-999a68f1d099 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:34.567563 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c76c29-8c91-4626-856b-8e55b188f42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:34.568295 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c76c29-8c91-4626-856b-8e55b188f42a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:34.568600 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c76c29-8c91-4626-856b-8e55b188f42a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:34.568907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c76c29-8c91-4626-856b-8e55b188f42a None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:34.569715 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c76c29-8c91-4626-856b-8e55b188f42a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:34.570027 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c76c29-8c91-4626-856b-8e55b188f42a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:34.571776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c76c29-8c91-4626-856b-8e55b188f42a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:34.592006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 183/370] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:45:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:34.597542 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c76c29-8c91-4626-856b-8e55b188f42a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:34.623321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89324e-18a3-4347-b572-3bdf4028e696 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:34.694708 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693d2bd-7291-4ecf-8e5e-aed620e0f96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:34.695521 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693d2bd-7291-4ecf-8e5e-aed620e0f96e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:34.695801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693d2bd-7291-4ecf-8e5e-aed620e0f96e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:34.696072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693d2bd-7291-4ecf-8e5e-aed620e0f96e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:34.715738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693d2bd-7291-4ecf-8e5e-aed620e0f96e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:34.725016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-7693d2bd-7291-4ecf-8e5e-aed620e0f96e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:45:34.725016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:34.725016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:34.725016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:34.725016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:34.725016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:34.725016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:34.725016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:34.725016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:34.725016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:34.725016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:34.725016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:34.725016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:34.725016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:34.725016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:34.725016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:34.725928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:34.725928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:34.725928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:34.725928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:34.725928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:34.725928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:34.725928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:34.725928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:34.725928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:34.725928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:34.725928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:34.725928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:34.725928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:34.725928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:34.725928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:34.725928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:34.726752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:34.726752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:34.726752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:34.726752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:34.726752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:34.726752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:34.726752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:34.726752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:34.726752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:34.726752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:34.726752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:45:34.726752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:34.726752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 188/371] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:45:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:34.735164 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693d2bd-7291-4ecf-8e5e-aed620e0f96e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:34.773001 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c76c29-8c91-4626-856b-8e55b188f42a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:34.781248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:34.808622 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:34.835921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3762f0d-6ae4-40db-a5c8-68fb26832ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:34.836658 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3762f0d-6ae4-40db-a5c8-68fb26832ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:34.837646 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3762f0d-6ae4-40db-a5c8-68fb26832ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:34.838268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3762f0d-6ae4-40db-a5c8-68fb26832ce0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:34.845401 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693d2bd-7291-4ecf-8e5e-aed620e0f96e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:34.880972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85f18d-0d60-457e-bc5d-767c16fc17f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:34.881687 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85f18d-0d60-457e-bc5d-767c16fc17f8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:34.881974 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85f18d-0d60-457e-bc5d-767c16fc17f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:34.882246 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85f18d-0d60-457e-bc5d-767c16fc17f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:34.883023 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85f18d-0d60-457e-bc5d-767c16fc17f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:34.883331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85f18d-0d60-457e-bc5d-767c16fc17f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:34.885239 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85f18d-0d60-457e-bc5d-767c16fc17f8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:34.894644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 189/372] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:34.900632 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85f18d-0d60-457e-bc5d-767c16fc17f8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:34.945386 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3762f0d-6ae4-40db-a5c8-68fb26832ce0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:34.954148 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3762f0d-6ae4-40db-a5c8-68fb26832ce0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:35.014903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85f18d-0d60-457e-bc5d-767c16fc17f8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:35.023082 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:35.054076 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a2869-a8fb-4c97-8e4f-28140efd7135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:35.054764 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a2869-a8fb-4c97-8e4f-28140efd7135 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:35.055053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a2869-a8fb-4c97-8e4f-28140efd7135 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:35.055337 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a2869-a8fb-4c97-8e4f-28140efd7135 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:35.131473 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a2869-a8fb-4c97-8e4f-28140efd7135 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:35.132967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a2869-a8fb-4c97-8e4f-28140efd7135 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:35.248423 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 190/373] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:45:35.256082 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a2869-a8fb-4c97-8e4f-28140efd7135 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:35.259323 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:35.288304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 184/374] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:45:35.295633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3762f0d-6ae4-40db-a5c8-68fb26832ce0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:35.324621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:35.396627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a2869-a8fb-4c97-8e4f-28140efd7135 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:35.443572 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb4d17-73df-4cb9-b26f-c3dcf6d35cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:35.444734 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb4d17-73df-4cb9-b26f-c3dcf6d35cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:35.445849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb4d17-73df-4cb9-b26f-c3dcf6d35cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:35.446420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb4d17-73df-4cb9-b26f-c3dcf6d35cd8 None admin] PATH_INFO: `/v3/projects/417f1f7349214e82b739864de0303f59` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:35.453104 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb4d17-73df-4cb9-b26f-c3dcf6d35cd8 None admin] RBAC: Authorizing `identity:get_project(project_id=417f1f7349214e82b739864de0303f59)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:35.454773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb4d17-73df-4cb9-b26f-c3dcf6d35cd8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:35.456657 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 191/375] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:35 2022] GET /identity/v3/projects/417f1f7349214e82b739864de0303f59 => generated 344 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:35.463733 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb4d17-73df-4cb9-b26f-c3dcf6d35cd8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:35.503011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3762f0d-6ae4-40db-a5c8-68fb26832ce0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:35.554216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3239c5-aeea-4fae-905f-c319d4cf6179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:35.555362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3239c5-aeea-4fae-905f-c319d4cf6179 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:35.555874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3239c5-aeea-4fae-905f-c319d4cf6179 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:35.556293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3239c5-aeea-4fae-905f-c319d4cf6179 None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:35.559488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3239c5-aeea-4fae-905f-c319d4cf6179 None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:35.566174 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3239c5-aeea-4fae-905f-c319d4cf6179 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:35.568443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 185/376] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:35 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:35.592934 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3239c5-aeea-4fae-905f-c319d4cf6179 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:35.626934 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:35.680432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb4d17-73df-4cb9-b26f-c3dcf6d35cd8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:35.706299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:35.736199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155af9e-30d4-4dbf-b373-416ada204994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:35.737079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155af9e-30d4-4dbf-b373-416ada204994 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:35.737364 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155af9e-30d4-4dbf-b373-416ada204994 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:35.737643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155af9e-30d4-4dbf-b373-416ada204994 None admin] PATH_INFO: `/v3/groups/578672aa776448f8a45473865665777c` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:35.740903 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3239c5-aeea-4fae-905f-c319d4cf6179 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:35.765603 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155af9e-30d4-4dbf-b373-416ada204994 None admin] RBAC: Authorizing `identity:get_group(group_id=578672aa776448f8a45473865665777c)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:35.768211 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155af9e-30d4-4dbf-b373-416ada204994 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:35.772084 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 192/377] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Jun 16 18:45:35 2022] GET /identity/v3/groups/578672aa776448f8a45473865665777c => generated 229 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:35.782267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155af9e-30d4-4dbf-b373-416ada204994 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:35.859371 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a34cb-4879-4152-8e7c-951ad8456680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:35.860760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a34cb-4879-4152-8e7c-951ad8456680 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:35.861245 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a34cb-4879-4152-8e7c-951ad8456680 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:35.861725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a34cb-4879-4152-8e7c-951ad8456680 None admin] PATH_INFO: `/v3/groups/859c364715ee47fabc3a3027a878e3d9` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:35.867610 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a34cb-4879-4152-8e7c-951ad8456680 None admin] RBAC: Authorizing `identity:get_group(group_id=859c364715ee47fabc3a3027a878e3d9)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:35.873056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a34cb-4879-4152-8e7c-951ad8456680 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:35.874178 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 186/378] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Jun 16 18:45:35 2022] GET /identity/v3/groups/859c364715ee47fabc3a3027a878e3d9 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:35.879730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a34cb-4879-4152-8e7c-951ad8456680 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:36.026312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155af9e-30d4-4dbf-b373-416ada204994 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:36.126415 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a5a0c-bcac-4f49-863f-6c808635c68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:36.129303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a5a0c-bcac-4f49-863f-6c808635c68d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:36.130332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a5a0c-bcac-4f49-863f-6c808635c68d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:36.131538 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a5a0c-bcac-4f49-863f-6c808635c68d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:36.132426 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a5a0c-bcac-4f49-863f-6c808635c68d None admin] RBAC: Adding query filter params (group.id=578672aa776448f8a45473865665777c, role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=417f1f7349214e82b739864de0303f59) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:36.132746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a5a0c-bcac-4f49-863f-6c808635c68d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:36.134297 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a5a0c-bcac-4f49-863f-6c808635c68d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:36.136731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:36.144645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a34cb-4879-4152-8e7c-951ad8456680 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:36.145382 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 193/379] 10.222.0.70 () {56 vars in 1376 bytes} [Thu Jun 16 18:45:35 2022] GET /identity/v3/role_assignments?group.id=578672aa776448f8a45473865665777c&scope.project.id=417f1f7349214e82b739864de0303f59&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 260 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:36.146710 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362663f-eb7a-40c7-9bd8-2b5b88957078 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:36.147021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362663f-eb7a-40c7-9bd8-2b5b88957078 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:36.147285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362663f-eb7a-40c7-9bd8-2b5b88957078 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:36.154606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 194/380] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:36.154819 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049be38-43fb-4599-9d53-285c6c5f55d9 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:36.155053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049be38-43fb-4599-9d53-285c6c5f55d9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:36.155268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049be38-43fb-4599-9d53-285c6c5f55d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:36.187604 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e049be38-43fb-4599-9d53-285c6c5f55d9 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:36.188827 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e049be38-43fb-4599-9d53-285c6c5f55d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:36.230423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f94d4d-0d55-491e-812a-a88dc00b6d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:36.231130 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f94d4d-0d55-491e-812a-a88dc00b6d86 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:36.231414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f94d4d-0d55-491e-812a-a88dc00b6d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:36.231692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f94d4d-0d55-491e-812a-a88dc00b6d86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:36.232538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f94d4d-0d55-491e-812a-a88dc00b6d86 None admin] RBAC: Adding query filter params (group.id=859c364715ee47fabc3a3027a878e3d9, role.id=662cb9b5dff94cdb83f292ad20d1d430, scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:36.232849 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f94d4d-0d55-491e-812a-a88dc00b6d86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:36.234398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f94d4d-0d55-491e-812a-a88dc00b6d86 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:36.250417 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 187/381] 10.222.0.70 () {56 vars in 1376 bytes} [Thu Jun 16 18:45:35 2022] GET /identity/v3/role_assignments?group.id=859c364715ee47fabc3a3027a878e3d9&scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df&role.id=662cb9b5dff94cdb83f292ad20d1d430 => generated 260 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:36.317514 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 195/382] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:36.352124 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f94d4d-0d55-491e-812a-a88dc00b6d86 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:36.374286 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:36.474618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5a1a1-d73d-4522-8b29-fc8723061abb None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:36.474963 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5a1a1-d73d-4522-8b29-fc8723061abb None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:36.475203 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5a1a1-d73d-4522-8b29-fc8723061abb None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:36.476813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 196/383] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:36.485012 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937de5c-248c-4c56-b75d-9c3e0cc97339 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:36.485332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937de5c-248c-4c56-b75d-9c3e0cc97339 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:36.485567 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937de5c-248c-4c56-b75d-9c3e0cc97339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:36.501755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f94d4d-0d55-491e-812a-a88dc00b6d86 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:36.527691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4937de5c-248c-4c56-b75d-9c3e0cc97339 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:36.528892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4937de5c-248c-4c56-b75d-9c3e0cc97339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:36.545250 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a9e70-30ae-424c-927c-cb5bc45263a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:36.546491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a9e70-30ae-424c-927c-cb5bc45263a0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:36.547015 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a9e70-30ae-424c-927c-cb5bc45263a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:36.547518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a9e70-30ae-424c-927c-cb5bc45263a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:36.564648 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a9e70-30ae-424c-927c-cb5bc45263a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:36.573757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ba3a9e70-30ae-424c-927c-cb5bc45263a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:36.573757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:36.573757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:36.573757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:36.573757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:36.573757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:36.573757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:36.573757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:36.573757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:36.573757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:36.573757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:36.573757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:36.573757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:36.573757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:36.573757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:36.573757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:36.574706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:36.574706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:36.574706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:36.574706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:36.574706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:36.574706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:36.574706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:36.574706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:36.574706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:36.574706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:36.574706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:36.574706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:36.574706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:36.574706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:36.574706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:36.574706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:36.575596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:36.575596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:36.575596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:36.575596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:36.575596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:36.575596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:36.575596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:36.575596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:36.575596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:36.575596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:36.575596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:36.575596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:36.575596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 188/384] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:36.583948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a9e70-30ae-424c-927c-cb5bc45263a0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:36.651311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 197/385] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:36.676273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937de5c-248c-4c56-b75d-9c3e0cc97339 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:36.725072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:36.754940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a9e70-30ae-424c-927c-cb5bc45263a0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:36.795574 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:36.807827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a93b28-a7fd-4bba-910e-6ad7f96982c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:36.808814 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a93b28-a7fd-4bba-910e-6ad7f96982c7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:36.809260 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a93b28-a7fd-4bba-910e-6ad7f96982c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:36.809720 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a93b28-a7fd-4bba-910e-6ad7f96982c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:36.810650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a93b28-a7fd-4bba-910e-6ad7f96982c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:36.811096 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a93b28-a7fd-4bba-910e-6ad7f96982c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:36.813907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a93b28-a7fd-4bba-910e-6ad7f96982c7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:36.827228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 189/386] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:36.829524 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76404ab6-44bc-4545-9f21-43353717f1e6 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:36.829962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76404ab6-44bc-4545-9f21-43353717f1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:36.834888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76404ab6-44bc-4545-9f21-43353717f1e6 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:36.840396 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 190/387] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:36 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:36.841763 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76404ab6-44bc-4545-9f21-43353717f1e6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:36.858075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937de5c-248c-4c56-b75d-9c3e0cc97339 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:36.904733 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7adab-9d8c-4cf2-a910-2fedb49015cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:36.905430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7adab-9d8c-4cf2-a910-2fedb49015cf None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:36.905699 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7adab-9d8c-4cf2-a910-2fedb49015cf None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:36.905978 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7adab-9d8c-4cf2-a910-2fedb49015cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:36.921088 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7adab-9d8c-4cf2-a910-2fedb49015cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:36.932676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-3ca7adab-9d8c-4cf2-a910-2fedb49015cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:36.932676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:36.932676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:36.932676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:36.932676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:36.932676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:36.932676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:36.932676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:36.932676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:36.932676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:36.932676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:36.932676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:36.932676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:36.932676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:36.932676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:36.932676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:36.933618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:36.933618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:36.933618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:36.933618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:36.933618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:36.933618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:36.933618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:36.933618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:36.933618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:36.933618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:36.933618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:36.933618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:36.933618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:36.933618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:36.933618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:36.933618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:36.934562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:36.934562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:36.934562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:36.934562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:36.934562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:36.934562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:36.934562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:36.934562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:36.934562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:36.934562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:36.934562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:36.934562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:36.934562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 198/388] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:45:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:45:36.935308 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee592f84-38cf-47af-9f05-3eac9fa247d4 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:36.935848 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee592f84-38cf-47af-9f05-3eac9fa247d4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:36.935848 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee592f84-38cf-47af-9f05-3eac9fa247d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:36.957406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ee592f84-38cf-47af-9f05-3eac9fa247d4 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:36.958683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ee592f84-38cf-47af-9f05-3eac9fa247d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:36.994645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76404ab6-44bc-4545-9f21-43353717f1e6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:37.039977 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f43f1b-13a8-4579-ac35-c70a5f7b94c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:37.041135 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f43f1b-13a8-4579-ac35-c70a5f7b94c3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:37.041602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f43f1b-13a8-4579-ac35-c70a5f7b94c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:37.041884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f43f1b-13a8-4579-ac35-c70a5f7b94c3 None admin] PATH_INFO: `/v3/groups/859c364715ee47fabc3a3027a878e3d9` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:37.043470 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f43f1b-13a8-4579-ac35-c70a5f7b94c3 None admin] RBAC: Authorizing `identity:get_group(group_id=859c364715ee47fabc3a3027a878e3d9)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:37.044913 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f43f1b-13a8-4579-ac35-c70a5f7b94c3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:37.047591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 191/389] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Jun 16 18:45:36 2022] GET /identity/v3/groups/859c364715ee47fabc3a3027a878e3d9 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:37.048373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f43f1b-13a8-4579-ac35-c70a5f7b94c3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:37.051810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:37.072120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 199/390] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:37.073016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee592f84-38cf-47af-9f05-3eac9fa247d4 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:37.134797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:37.199447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f43f1b-13a8-4579-ac35-c70a5f7b94c3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:37.205976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee592f84-38cf-47af-9f05-3eac9fa247d4 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:37.238924 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf7e15-517a-4696-baef-871dfa7ee92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:37.239600 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf7e15-517a-4696-baef-871dfa7ee92f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:37.239881 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf7e15-517a-4696-baef-871dfa7ee92f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:37.240158 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf7e15-517a-4696-baef-871dfa7ee92f None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:37.240902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf7e15-517a-4696-baef-871dfa7ee92f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:37.241203 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf7e15-517a-4696-baef-871dfa7ee92f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:37.242563 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf7e15-517a-4696-baef-871dfa7ee92f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:37.251196 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc49ef-259f-44fc-907d-b6c9085de70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:37.252064 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc49ef-259f-44fc-907d-b6c9085de70f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:37.252345 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc49ef-259f-44fc-907d-b6c9085de70f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:37.252620 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc49ef-259f-44fc-907d-b6c9085de70f None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:37.252838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 192/391] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:45:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:37.253553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf7e15-517a-4696-baef-871dfa7ee92f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:37.253825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc49ef-259f-44fc-907d-b6c9085de70f None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:37.255486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc49ef-259f-44fc-907d-b6c9085de70f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:37.258693 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 200/392] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:37 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:37.260021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc49ef-259f-44fc-907d-b6c9085de70f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:37.328422 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:37.374219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf7e15-517a-4696-baef-871dfa7ee92f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:37.396525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc49ef-259f-44fc-907d-b6c9085de70f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:37.417019 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:37.419274 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c827c1-a09c-4a3d-ad15-d919d943300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:37.420078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c827c1-a09c-4a3d-ad15-d919d943300a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:37.420360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c827c1-a09c-4a3d-ad15-d919d943300a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:37.420634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c827c1-a09c-4a3d-ad15-d919d943300a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:37.436572 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c827c1-a09c-4a3d-ad15-d919d943300a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:37.438442 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891c4c0-f738-48bc-b311-28bc2dbb09b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:37.439326 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891c4c0-f738-48bc-b311-28bc2dbb09b9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:37.439606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891c4c0-f738-48bc-b311-28bc2dbb09b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:37.439933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891c4c0-f738-48bc-b311-28bc2dbb09b9 None admin] PATH_INFO: `/v3/groups/859c364715ee47fabc3a3027a878e3d9` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:37.441156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2891c4c0-f738-48bc-b311-28bc2dbb09b9 None admin] RBAC: Authorizing `identity:get_group(group_id=859c364715ee47fabc3a3027a878e3d9)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:37.442596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2891c4c0-f738-48bc-b311-28bc2dbb09b9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:37.444553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-a5c827c1-a09c-4a3d-ad15-d919d943300a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:37.444553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:37.444553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:37.444553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:37.444553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:37.444553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:37.444553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:37.444553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:37.444553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:37.444553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:37.444553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:37.444553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:37.444553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:37.444553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:37.444553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:37.444553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:37.446700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:37.446700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:37.446700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:37.446700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:37.446700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:37.446700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:37.446700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:37.446700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:37.446700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:37.446700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:37.446700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:37.446700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:37.446700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:37.446700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:37.446700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:37.446700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:37.448641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:37.448641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:37.448641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:37.448641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:37.448641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:37.448641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:37.448641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:37.448641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:37.448641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:37.448641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:37.448641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:37.448641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:37.449659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 201/393] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Jun 16 18:45:37 2022] GET /identity/v3/groups/859c364715ee47fabc3a3027a878e3d9 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:37.449659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891c4c0-f738-48bc-b311-28bc2dbb09b9 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:37.449889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 193/394] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:37.453936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c827c1-a09c-4a3d-ad15-d919d943300a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:37.598392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891c4c0-f738-48bc-b311-28bc2dbb09b9 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:37.609182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c827c1-a09c-4a3d-ad15-d919d943300a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:37.637490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:37.641474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fba0e-d8bd-433d-a681-0a5e7566404e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:37.642370 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fba0e-d8bd-433d-a681-0a5e7566404e None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:37.642680 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fba0e-d8bd-433d-a681-0a5e7566404e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:37.642961 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fba0e-d8bd-433d-a681-0a5e7566404e None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be/groups/859c364715ee47fabc3a3027a878e3d9/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:37.645183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fba0e-d8bd-433d-a681-0a5e7566404e None admin] RBAC: Authorizing `identity:create_grant(project_id=aa7a11e47f684403b0ebf54d869da5be, group_id=859c364715ee47fabc3a3027a878e3d9, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:37.646825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fba0e-d8bd-433d-a681-0a5e7566404e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:37.650487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bb16e-15b1-4b81-afae-30f8935bd553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:37.651221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bb16e-15b1-4b81-afae-30f8935bd553 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:37.651512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bb16e-15b1-4b81-afae-30f8935bd553 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:37.651801 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bb16e-15b1-4b81-afae-30f8935bd553 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:37.652573 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bb16e-15b1-4b81-afae-30f8935bd553 None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:37.652885 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bb16e-15b1-4b81-afae-30f8935bd553 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:37.654335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bb16e-15b1-4b81-afae-30f8935bd553 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:37.665398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 194/395] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:37.665882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 202/396] 10.222.0.70 () {58 vars in 1442 bytes} [Thu Jun 16 18:45:37 2022] PUT /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be/groups/859c364715ee47fabc3a3027a878e3d9/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:45:37.671403 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bb16e-15b1-4b81-afae-30f8935bd553 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:37.672960 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fba0e-d8bd-433d-a681-0a5e7566404e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:37.747534 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:37.803184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fba0e-d8bd-433d-a681-0a5e7566404e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:37.827074 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bb16e-15b1-4b81-afae-30f8935bd553 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:37.845158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f815898-e96d-4feb-9816-7cb9a571a53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:37.846190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f815898-e96d-4feb-9816-7cb9a571a53c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:37.846481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f815898-e96d-4feb-9816-7cb9a571a53c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:37.846805 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f815898-e96d-4feb-9816-7cb9a571a53c None admin] PATH_INFO: `/v3/groups/859c364715ee47fabc3a3027a878e3d9` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:37.848035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f815898-e96d-4feb-9816-7cb9a571a53c None admin] RBAC: Authorizing `identity:get_group(group_id=859c364715ee47fabc3a3027a878e3d9)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:37.849627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f815898-e96d-4feb-9816-7cb9a571a53c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:37.851685 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 203/397] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Jun 16 18:45:37 2022] GET /identity/v3/groups/859c364715ee47fabc3a3027a878e3d9 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:37.868928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f815898-e96d-4feb-9816-7cb9a571a53c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:37.872481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572d6cf-a5ee-4faf-8567-f0aa951bb7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:37.873369 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572d6cf-a5ee-4faf-8567-f0aa951bb7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:37.873659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572d6cf-a5ee-4faf-8567-f0aa951bb7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:37.873937 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572d6cf-a5ee-4faf-8567-f0aa951bb7a7 None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:37.875182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572d6cf-a5ee-4faf-8567-f0aa951bb7a7 None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:37.876626 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572d6cf-a5ee-4faf-8567-f0aa951bb7a7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:37.878647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 195/398] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:37 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:37.884678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572d6cf-a5ee-4faf-8567-f0aa951bb7a7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:38.002755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:38.060611 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572d6cf-a5ee-4faf-8567-f0aa951bb7a7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:38.101914 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f815898-e96d-4feb-9816-7cb9a571a53c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:38.109476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea27a9-83be-4854-b5b9-1ef00df671b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:38.110429 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea27a9-83be-4854-b5b9-1ef00df671b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:38.110742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea27a9-83be-4854-b5b9-1ef00df671b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:38.111063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea27a9-83be-4854-b5b9-1ef00df671b6 None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be/groups/859c364715ee47fabc3a3027a878e3d9/roles/662cb9b5dff94cdb83f292ad20d1d430` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:38.118080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea27a9-83be-4854-b5b9-1ef00df671b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa7a11e47f684403b0ebf54d869da5be, group_id=859c364715ee47fabc3a3027a878e3d9, role_id=662cb9b5dff94cdb83f292ad20d1d430)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:38.119852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea27a9-83be-4854-b5b9-1ef00df671b6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:38.120566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:38.137508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 196/399] 10.222.0.70 () {58 vars in 1442 bytes} [Thu Jun 16 18:45:37 2022] PUT /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be/groups/859c364715ee47fabc3a3027a878e3d9/roles/662cb9b5dff94cdb83f292ad20d1d430 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:45:38.154783 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556be194-e2a1-4837-8678-2b55a197abf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:38.156170 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556be194-e2a1-4837-8678-2b55a197abf5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:38.156750 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556be194-e2a1-4837-8678-2b55a197abf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:38.157042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556be194-e2a1-4837-8678-2b55a197abf5 None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:38.159436 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556be194-e2a1-4837-8678-2b55a197abf5 None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:38.160885 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556be194-e2a1-4837-8678-2b55a197abf5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:38.162818 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 204/400] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:37 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:38.169749 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea27a9-83be-4854-b5b9-1ef00df671b6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:38.298794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea27a9-83be-4854-b5b9-1ef00df671b6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:38.319778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:38.348192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af4dd5-1191-4271-99d0-12571b051ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:38.349567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af4dd5-1191-4271-99d0-12571b051ff3 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:38.350099 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af4dd5-1191-4271-99d0-12571b051ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:38.350636 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af4dd5-1191-4271-99d0-12571b051ff3 None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df/groups/859c364715ee47fabc3a3027a878e3d9/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:38.353092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af4dd5-1191-4271-99d0-12571b051ff3 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df, group_id=859c364715ee47fabc3a3027a878e3d9, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:38.354875 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af4dd5-1191-4271-99d0-12571b051ff3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:38.378281 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 197/401] 10.222.0.70 () {58 vars in 1442 bytes} [Thu Jun 16 18:45:38 2022] PUT /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df/groups/859c364715ee47fabc3a3027a878e3d9/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:45:38.999577 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9228a-dd4e-47f9-88fc-ab4607afd65d None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:39.000246 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9228a-dd4e-47f9-88fc-ab4607afd65d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:39.000728 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9228a-dd4e-47f9-88fc-ab4607afd65d None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:39.002684 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 205/402] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:39.008131 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf4fff-23ef-4541-81ba-8f7cf9276902 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:39.009701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf4fff-23ef-4541-81ba-8f7cf9276902 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:39.010200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf4fff-23ef-4541-81ba-8f7cf9276902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:39.033154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4aaf4fff-23ef-4541-81ba-8f7cf9276902 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:39.034762 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4aaf4fff-23ef-4541-81ba-8f7cf9276902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:39.053965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751465d-2088-4746-a27b-b0e8cb033eff None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:39.054299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751465d-2088-4746-a27b-b0e8cb033eff None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:39.054522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751465d-2088-4746-a27b-b0e8cb033eff None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:39.056865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 206/403] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:45:39.072706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcb805-c9f2-441d-840b-4423098b233d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:39.073065 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcb805-c9f2-441d-840b-4423098b233d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:39.073284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcb805-c9f2-441d-840b-4423098b233d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:39.095654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-bddcb805-c9f2-441d-840b-4423098b233d None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:39.097458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-bddcb805-c9f2-441d-840b-4423098b233d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:39.173004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 198/404] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:39.195372 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf4fff-23ef-4541-81ba-8f7cf9276902 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:39.245696 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 207/405] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:39.281305 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcb805-c9f2-441d-840b-4423098b233d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:39.341178 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf4fff-23ef-4541-81ba-8f7cf9276902 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:39.344548 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:39.392736 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b9c13-8f44-4ff2-9001-3ded2b5e6357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:39.393808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b9c13-8f44-4ff2-9001-3ded2b5e6357 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:39.394090 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b9c13-8f44-4ff2-9001-3ded2b5e6357 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:39.394475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b9c13-8f44-4ff2-9001-3ded2b5e6357 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:39.414406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b9c13-8f44-4ff2-9001-3ded2b5e6357 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:39.428306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-d04b9c13-8f44-4ff2-9001-3ded2b5e6357 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:45:39.428306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:39.428306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:39.428306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:39.428306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:39.428306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:39.428306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:39.428306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:39.428306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:39.428306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:39.428306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:39.428306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:39.428306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:39.428306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:39.428306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:39.428306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:39.429233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:39.429233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:39.429233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:39.429233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:39.429233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:39.429233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:39.429233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:39.429233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:39.429233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:39.429233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:39.429233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:39.429233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:39.429233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:39.429233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:39.429233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:39.429233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:39.430022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:39.430022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:39.430022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:39.430022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:39.430022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:39.430022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:39.430022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:39.430022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:39.430022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:39.430022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:39.430022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:45:39.430022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:39.431958 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 199/406] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:45:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:45:39.441270 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b9c13-8f44-4ff2-9001-3ded2b5e6357 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:39.488328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcb805-c9f2-441d-840b-4423098b233d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:39.507917 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:39.557325 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f1742-13a0-4034-9718-898c327de985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:39.558140 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f1742-13a0-4034-9718-898c327de985 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:39.558418 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f1742-13a0-4034-9718-898c327de985 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:39.558717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f1742-13a0-4034-9718-898c327de985 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:39.581404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f1742-13a0-4034-9718-898c327de985 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:39.589836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-e55f1742-13a0-4034-9718-898c327de985 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:39.589836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:39.589836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:39.589836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:39.589836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:39.589836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:39.589836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:39.589836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:39.589836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:39.589836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:39.589836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:39.589836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:39.589836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:39.589836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:39.589836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:39.589836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:39.590740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:39.590740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:39.590740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:39.590740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:39.590740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:39.590740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:39.590740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:39.590740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:39.590740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:39.590740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:39.590740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:39.590740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:39.590740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:39.590740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:39.590740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:39.590740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:39.591558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:39.591558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:39.591558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:39.591558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:39.591558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:39.591558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:39.591558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:39.591558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:39.591558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:39.591558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:39.591558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:39.591558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:39.596137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 208/407] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:45:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:39.603752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f1742-13a0-4034-9718-898c327de985 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:39.620175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b9c13-8f44-4ff2-9001-3ded2b5e6357 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:39.663297 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43332f-2dc8-4cfb-b587-27e3a07945bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:39.664351 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43332f-2dc8-4cfb-b587-27e3a07945bc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:39.665375 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43332f-2dc8-4cfb-b587-27e3a07945bc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:39.665877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43332f-2dc8-4cfb-b587-27e3a07945bc None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:39.666933 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43332f-2dc8-4cfb-b587-27e3a07945bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:39.667878 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43332f-2dc8-4cfb-b587-27e3a07945bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:39.669936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43332f-2dc8-4cfb-b587-27e3a07945bc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:39.688033 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 200/408] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:39.689788 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43332f-2dc8-4cfb-b587-27e3a07945bc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:39.724142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:39.806285 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43332f-2dc8-4cfb-b587-27e3a07945bc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:39.838180 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f1742-13a0-4034-9718-898c327de985 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:39.842754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5d91f-2d38-4eb4-bba2-e13497dda6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:39.843881 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5d91f-2d38-4eb4-bba2-e13497dda6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:39.844351 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5d91f-2d38-4eb4-bba2-e13497dda6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:39.844810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5d91f-2d38-4eb4-bba2-e13497dda6d6 None admin] PATH_INFO: `/v3/groups/578672aa776448f8a45473865665777c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:39.846442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce5d91f-2d38-4eb4-bba2-e13497dda6d6 None admin] RBAC: Authorizing `identity:get_group(group_id=578672aa776448f8a45473865665777c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:39.848083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce5d91f-2d38-4eb4-bba2-e13497dda6d6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:39.854720 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 201/409] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Jun 16 18:45:39 2022] GET /identity/v3/groups/578672aa776448f8a45473865665777c => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:39.856245 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5d91f-2d38-4eb4-bba2-e13497dda6d6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:39.889204 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa822b-db76-4dbd-8305-179d7a1c137c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:39.889879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa822b-db76-4dbd-8305-179d7a1c137c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:39.890397 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa822b-db76-4dbd-8305-179d7a1c137c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:39.890524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa822b-db76-4dbd-8305-179d7a1c137c None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:39.891194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa822b-db76-4dbd-8305-179d7a1c137c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:39.891554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa822b-db76-4dbd-8305-179d7a1c137c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:39.892878 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa822b-db76-4dbd-8305-179d7a1c137c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:39.906453 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 209/410] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:45:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:39.919638 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa822b-db76-4dbd-8305-179d7a1c137c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:39.966942 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:39.979338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:39.979805 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5d91f-2d38-4eb4-bba2-e13497dda6d6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:40.024931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da94ed3-dd02-47de-8c8c-8780868f2a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:40.026817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da94ed3-dd02-47de-8c8c-8780868f2a8e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:40.027299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da94ed3-dd02-47de-8c8c-8780868f2a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:40.027763 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da94ed3-dd02-47de-8c8c-8780868f2a8e None admin] PATH_INFO: `/v3/projects/417f1f7349214e82b739864de0303f59` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:40.029223 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da94ed3-dd02-47de-8c8c-8780868f2a8e None admin] RBAC: Authorizing `identity:get_project(project_id=417f1f7349214e82b739864de0303f59)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:40.030892 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da94ed3-dd02-47de-8c8c-8780868f2a8e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:40.037555 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 202/411] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:39 2022] GET /identity/v3/projects/417f1f7349214e82b739864de0303f59 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:40.039105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da94ed3-dd02-47de-8c8c-8780868f2a8e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:40.129467 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa822b-db76-4dbd-8305-179d7a1c137c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:40.168057 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a70695-471f-4ab2-becd-583be56d4f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:40.168927 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a70695-471f-4ab2-becd-583be56d4f2a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:40.169217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a70695-471f-4ab2-becd-583be56d4f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:40.169499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a70695-471f-4ab2-becd-583be56d4f2a None admin] PATH_INFO: `/v3/groups/859c364715ee47fabc3a3027a878e3d9` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:40.170856 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a70695-471f-4ab2-becd-583be56d4f2a None admin] RBAC: Authorizing `identity:get_group(group_id=859c364715ee47fabc3a3027a878e3d9)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:40.172304 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a70695-471f-4ab2-becd-583be56d4f2a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:40.173918 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 210/412] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Jun 16 18:45:39 2022] GET /identity/v3/groups/859c364715ee47fabc3a3027a878e3d9 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:40.179945 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a70695-471f-4ab2-becd-583be56d4f2a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:40.221936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da94ed3-dd02-47de-8c8c-8780868f2a8e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:40.269096 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb072e58-3bcf-415e-aa09-ef4baad499c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:40.270368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb072e58-3bcf-415e-aa09-ef4baad499c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:40.270872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb072e58-3bcf-415e-aa09-ef4baad499c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:40.271453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb072e58-3bcf-415e-aa09-ef4baad499c7 None admin] PATH_INFO: `/v3/projects/417f1f7349214e82b739864de0303f59/groups/578672aa776448f8a45473865665777c/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:40.273613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb072e58-3bcf-415e-aa09-ef4baad499c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=417f1f7349214e82b739864de0303f59, group_id=578672aa776448f8a45473865665777c, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:40.276215 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb072e58-3bcf-415e-aa09-ef4baad499c7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:40.298767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 203/413] 10.222.0.70 () {58 vars in 1442 bytes} [Thu Jun 16 18:45:40 2022] PUT /identity/v3/projects/417f1f7349214e82b739864de0303f59/groups/578672aa776448f8a45473865665777c/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:45:40.379015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:40.383110 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a70695-471f-4ab2-becd-583be56d4f2a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:40.436141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1e018-1732-4330-9fb8-4c7bdd08d5eb None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:40.436448 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1e018-1732-4330-9fb8-4c7bdd08d5eb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:40.436717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1e018-1732-4330-9fb8-4c7bdd08d5eb None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:40.437346 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b96ccb-ff3f-493f-93d2-d13c019c434c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:40.438310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 204/414] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:45:40.438413 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b96ccb-ff3f-493f-93d2-d13c019c434c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:40.438485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b96ccb-ff3f-493f-93d2-d13c019c434c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:40.438791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b96ccb-ff3f-493f-93d2-d13c019c434c None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:40.440043 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b96ccb-ff3f-493f-93d2-d13c019c434c None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:40.441495 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b96ccb-ff3f-493f-93d2-d13c019c434c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:40.443568 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 211/415] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:40 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:40.450039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1e018-1732-4330-9fb8-4c7bdd08d5eb None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:40.452287 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd2d61-13bc-47bb-bb77-e50ceb0085c3 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:40.452587 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd2d61-13bc-47bb-bb77-e50ceb0085c3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:40.452868 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd2d61-13bc-47bb-bb77-e50ceb0085c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:40.481694 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-92dd2d61-13bc-47bb-bb77-e50ceb0085c3 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:40.482924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-92dd2d61-13bc-47bb-bb77-e50ceb0085c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:40.496991 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:40.563637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 212/416] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:40.591434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd2d61-13bc-47bb-bb77-e50ceb0085c3 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:40.596927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1e018-1732-4330-9fb8-4c7bdd08d5eb None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:40.634203 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82acea-5c11-4822-918c-3a2b2b6ca327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:40.635648 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82acea-5c11-4822-918c-3a2b2b6ca327 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:40.636180 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82acea-5c11-4822-918c-3a2b2b6ca327 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:40.636667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82acea-5c11-4822-918c-3a2b2b6ca327 None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df/groups/859c364715ee47fabc3a3027a878e3d9/roles/662cb9b5dff94cdb83f292ad20d1d430` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:40.639322 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82acea-5c11-4822-918c-3a2b2b6ca327 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df, group_id=859c364715ee47fabc3a3027a878e3d9, role_id=662cb9b5dff94cdb83f292ad20d1d430)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:40.640996 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82acea-5c11-4822-918c-3a2b2b6ca327 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:40.666892 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 205/417] 10.222.0.70 () {58 vars in 1442 bytes} [Thu Jun 16 18:45:40 2022] PUT /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df/groups/859c364715ee47fabc3a3027a878e3d9/roles/662cb9b5dff94cdb83f292ad20d1d430 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:45:40.730637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:40.734194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd2d61-13bc-47bb-bb77-e50ceb0085c3 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:40.786869 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6f7fd-4958-44b0-bf92-04b69c01af9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:40.787685 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6f7fd-4958-44b0-bf92-04b69c01af9c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:40.787971 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6f7fd-4958-44b0-bf92-04b69c01af9c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:40.788248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6f7fd-4958-44b0-bf92-04b69c01af9c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:40.805527 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6f7fd-4958-44b0-bf92-04b69c01af9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:40.814199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-02c6f7fd-4958-44b0-bf92-04b69c01af9c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:40.814199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:40.814199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:40.814199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:40.814199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:40.814199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:40.814199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:40.814199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:40.814199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:40.814199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:40.814199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:40.814199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:40.814199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:40.814199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:40.814199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:40.814199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:40.815168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:40.815168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:40.815168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:40.815168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:40.815168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:40.815168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:40.815168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:40.815168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:40.815168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:40.815168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:40.815168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:40.815168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:40.815168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:40.815168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:40.815168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:40.815168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:40.815952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:40.815952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:40.815952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:40.815952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:40.815952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:40.815952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:40.815952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:40.815952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:40.815952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:40.815952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:40.815952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:40.815952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:40.822092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82acea-5c11-4822-918c-3a2b2b6ca327 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:40.823333 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 213/418] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:40.943459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:40.964210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82acea-5c11-4822-918c-3a2b2b6ca327 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:41.010861 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02724e5f-925f-4829-80ec-773d7b8c5ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:41.011558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02724e5f-925f-4829-80ec-773d7b8c5ecf None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:41.011845 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02724e5f-925f-4829-80ec-773d7b8c5ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:41.012125 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02724e5f-925f-4829-80ec-773d7b8c5ecf None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:41.012884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02724e5f-925f-4829-80ec-773d7b8c5ecf None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:41.013191 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02724e5f-925f-4829-80ec-773d7b8c5ecf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:41.014562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02724e5f-925f-4829-80ec-773d7b8c5ecf None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:41.024664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 206/419] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:41.030144 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6f7fd-4958-44b0-bf92-04b69c01af9c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:41.196813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6f7fd-4958-44b0-bf92-04b69c01af9c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:41.253226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e71979-cfdb-4b85-9306-06fbc5114ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:41.253902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e71979-cfdb-4b85-9306-06fbc5114ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:41.254191 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e71979-cfdb-4b85-9306-06fbc5114ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:41.254473 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e71979-cfdb-4b85-9306-06fbc5114ec6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:41.260924 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f880976-98b8-4ce8-8789-767cd4b6dbcd None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:41.261410 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f880976-98b8-4ce8-8789-767cd4b6dbcd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:41.261483 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f880976-98b8-4ce8-8789-767cd4b6dbcd None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:41.262889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 207/420] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:41.272174 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fec573-60fa-4760-a5c5-1e835aad6cad None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:41.272465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fec573-60fa-4760-a5c5-1e835aad6cad None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:41.272748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fec573-60fa-4760-a5c5-1e835aad6cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:41.294899 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:41.312799 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-37fec573-60fa-4760-a5c5-1e835aad6cad None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:41.314326 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-37fec573-60fa-4760-a5c5-1e835aad6cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:41.378938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e71979-cfdb-4b85-9306-06fbc5114ec6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:41.384238 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e71979-cfdb-4b85-9306-06fbc5114ec6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:41.427927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 208/421] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 16 18:45:41.436252 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e94d2-dc89-42e7-b0d7-093f7aae2a10 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:41.436761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e94d2-dc89-42e7-b0d7-093f7aae2a10 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:41.437249 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e94d2-dc89-42e7-b0d7-093f7aae2a10 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:41.438763 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 209/422] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:41.444211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d972ad-8e63-4966-88d5-06055a7c373c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:41.444498 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d972ad-8e63-4966-88d5-06055a7c373c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:41.444778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d972ad-8e63-4966-88d5-06055a7c373c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:41.472140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-49d972ad-8e63-4966-88d5-06055a7c373c None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:41.473245 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-49d972ad-8e63-4966-88d5-06055a7c373c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:41.519881 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 214/423] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:41.521270 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e71979-cfdb-4b85-9306-06fbc5114ec6 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:41.549130 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 210/424] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:41.549989 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d972ad-8e63-4966-88d5-06055a7c373c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:41.553281 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:41.594059 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:41.652207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e71979-cfdb-4b85-9306-06fbc5114ec6 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:41.675761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d972ad-8e63-4966-88d5-06055a7c373c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:41.696470 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc24b2-ff58-420a-80b4-52cb97a66374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:41.698831 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc24b2-ff58-420a-80b4-52cb97a66374 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:41.699406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc24b2-ff58-420a-80b4-52cb97a66374 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:41.699882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc24b2-ff58-420a-80b4-52cb97a66374 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:41.714073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc24b2-ff58-420a-80b4-52cb97a66374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:41.719783 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a4da5-5121-42ea-8f9a-040c6068bd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:41.720757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a4da5-5121-42ea-8f9a-040c6068bd7d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:41.721041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a4da5-5121-42ea-8f9a-040c6068bd7d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:41.721314 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a4da5-5121-42ea-8f9a-040c6068bd7d None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:41.722596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a4da5-5121-42ea-8f9a-040c6068bd7d None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:41.724176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a4da5-5121-42ea-8f9a-040c6068bd7d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:41.724683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-abfc24b2-ff58-420a-80b4-52cb97a66374 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:41.724683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:41.724683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:41.724683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:41.724683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:41.724683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:41.724683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:41.724683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:41.724683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:41.724683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:41.724683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:41.724683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:41.724683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:41.724683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:41.724683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:41.724683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:41.725547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:41.725547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:41.725547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:41.725547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:41.725547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:41.725547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:41.725547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:41.725547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:41.725547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:41.725547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:41.725547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:41.725547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:41.725547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:41.725547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:41.725547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:41.725547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:41.726350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:41.726350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:41.726350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:41.726350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:41.726350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:41.726350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:41.726350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:41.726350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:41.726350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:41.726350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:41.726350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:41.726350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:41.727907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 211/425] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:41 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:41.727907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a4da5-5121-42ea-8f9a-040c6068bd7d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:41.734343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 215/426] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:45:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:41.735627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc24b2-ff58-420a-80b4-52cb97a66374 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:41.818012 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:41.851489 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a4da5-5121-42ea-8f9a-040c6068bd7d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:41.863534 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc24b2-ff58-420a-80b4-52cb97a66374 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:41.887654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:41.897494 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc138993-8eb6-44ec-8b07-47a416ec4172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:41.898710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc138993-8eb6-44ec-8b07-47a416ec4172 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:41.899789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc138993-8eb6-44ec-8b07-47a416ec4172 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:41.900032 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc138993-8eb6-44ec-8b07-47a416ec4172 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:41.910193 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38876b-1ae8-4cc6-af46-7f85d7ef685a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:41.911407 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38876b-1ae8-4cc6-af46-7f85d7ef685a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:41.911886 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38876b-1ae8-4cc6-af46-7f85d7ef685a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:41.912354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38876b-1ae8-4cc6-af46-7f85d7ef685a None admin] PATH_INFO: `/v3/groups/859c364715ee47fabc3a3027a878e3d9` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:41.913002 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc138993-8eb6-44ec-8b07-47a416ec4172 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:41.914005 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b38876b-1ae8-4cc6-af46-7f85d7ef685a None admin] RBAC: Authorizing `identity:get_group(group_id=859c364715ee47fabc3a3027a878e3d9)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:41.916203 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b38876b-1ae8-4cc6-af46-7f85d7ef685a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:41.918131 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 216/427] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Jun 16 18:45:41 2022] GET /identity/v3/groups/859c364715ee47fabc3a3027a878e3d9 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:41.919213 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38876b-1ae8-4cc6-af46-7f85d7ef685a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:41.921528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-cc138993-8eb6-44ec-8b07-47a416ec4172 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:41.921528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:41.921528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:41.921528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:41.921528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:41.921528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:41.921528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:41.921528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:41.921528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:41.921528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:41.921528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:41.921528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:41.921528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:41.921528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:41.921528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:41.921528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:41.922436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:41.922436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:41.922436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:41.922436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:41.922436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:41.922436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:41.922436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:41.922436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:41.922436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:41.922436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:41.922436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:41.922436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:41.922436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:41.922436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:41.922436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:41.922436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:41.923268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:41.923268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:41.923268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:41.923268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:41.923268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:41.923268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:41.923268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:41.923268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:41.923268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:41.923268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:41.923268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:45:41.923268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:41.923268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 212/428] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:41.928132 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc138993-8eb6-44ec-8b07-47a416ec4172 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:42.042757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38876b-1ae8-4cc6-af46-7f85d7ef685a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:42.074833 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc138993-8eb6-44ec-8b07-47a416ec4172 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:42.080630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b1d68-fa8f-4e63-af6d-db243fe6cda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:42.081565 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b1d68-fa8f-4e63-af6d-db243fe6cda7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:42.082049 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b1d68-fa8f-4e63-af6d-db243fe6cda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:42.082523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b1d68-fa8f-4e63-af6d-db243fe6cda7 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:42.083533 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b1d68-fa8f-4e63-af6d-db243fe6cda7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:42.084040 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b1d68-fa8f-4e63-af6d-db243fe6cda7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:42.085616 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b1d68-fa8f-4e63-af6d-db243fe6cda7 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:42.094631 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 217/429] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:45:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:42.099878 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b1d68-fa8f-4e63-af6d-db243fe6cda7 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:42.111276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:42.116898 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeea77f-d36a-41ef-8659-8b5d13edd193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:42.118768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeea77f-d36a-41ef-8659-8b5d13edd193 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:42.119260 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeea77f-d36a-41ef-8659-8b5d13edd193 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:42.120188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeea77f-d36a-41ef-8659-8b5d13edd193 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:42.121453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeea77f-d36a-41ef-8659-8b5d13edd193 None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:42.121963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeea77f-d36a-41ef-8659-8b5d13edd193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:42.124192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeea77f-d36a-41ef-8659-8b5d13edd193 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:42.137541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 213/430] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:45:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:42.138748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeea77f-d36a-41ef-8659-8b5d13edd193 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:42.157208 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:42.218317 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b1d68-fa8f-4e63-af6d-db243fe6cda7 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:42.255760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c9846-84b6-4ca3-b0b7-d79d8fdc5b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:42.256474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c9846-84b6-4ca3-b0b7-d79d8fdc5b78 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:42.256832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c9846-84b6-4ca3-b0b7-d79d8fdc5b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:42.257182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c9846-84b6-4ca3-b0b7-d79d8fdc5b78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:42.258185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c9846-84b6-4ca3-b0b7-d79d8fdc5b78 None admin] RBAC: Adding query filter params (group.id=859c364715ee47fabc3a3027a878e3d9, role.id=cf55e7c6398f45efaf10dec4fe153f8a, scope.project.id=aa7a11e47f684403b0ebf54d869da5be) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:42.258651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c9846-84b6-4ca3-b0b7-d79d8fdc5b78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:42.260319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c9846-84b6-4ca3-b0b7-d79d8fdc5b78 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:42.274956 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 218/431] 10.222.0.70 () {56 vars in 1376 bytes} [Thu Jun 16 18:45:42 2022] GET /identity/v3/role_assignments?group.id=859c364715ee47fabc3a3027a878e3d9&scope.project.id=aa7a11e47f684403b0ebf54d869da5be&role.id=cf55e7c6398f45efaf10dec4fe153f8a => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:42.276298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c9846-84b6-4ca3-b0b7-d79d8fdc5b78 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:42.283565 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeea77f-d36a-41ef-8659-8b5d13edd193 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:42.343070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1952b2f-8ee5-4413-bd21-b23e68af3092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:42.344172 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1952b2f-8ee5-4413-bd21-b23e68af3092 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:42.344767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1952b2f-8ee5-4413-bd21-b23e68af3092 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:42.345227 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1952b2f-8ee5-4413-bd21-b23e68af3092 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:42.425545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:42.437395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1952b2f-8ee5-4413-bd21-b23e68af3092 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:42.439440 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1952b2f-8ee5-4413-bd21-b23e68af3092 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:42.515676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c9846-84b6-4ca3-b0b7-d79d8fdc5b78 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:42.537326 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:42.567571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 214/432] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:42.573764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1952b2f-8ee5-4413-bd21-b23e68af3092 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:42.589456 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41daf87c-4d0a-45f7-9258-b95c9814ef02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:42.590324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41daf87c-4d0a-45f7-9258-b95c9814ef02 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:42.590712 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41daf87c-4d0a-45f7-9258-b95c9814ef02 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:42.591063 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41daf87c-4d0a-45f7-9258-b95c9814ef02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:42.681175 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41daf87c-4d0a-45f7-9258-b95c9814ef02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:42.683161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41daf87c-4d0a-45f7-9258-b95c9814ef02 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:42.695268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:42.703202 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1952b2f-8ee5-4413-bd21-b23e68af3092 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:42.756132 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253a201-7e35-4a7b-86bb-6f5e0822aa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:42.757014 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253a201-7e35-4a7b-86bb-6f5e0822aa26 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:42.757299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253a201-7e35-4a7b-86bb-6f5e0822aa26 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:42.757575 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253a201-7e35-4a7b-86bb-6f5e0822aa26 None admin] PATH_INFO: `/v3/projects/aa7a11e47f684403b0ebf54d869da5be` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:42.759342 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253a201-7e35-4a7b-86bb-6f5e0822aa26 None admin] RBAC: Authorizing `identity:get_project(project_id=aa7a11e47f684403b0ebf54d869da5be)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:42.761124 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253a201-7e35-4a7b-86bb-6f5e0822aa26 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:42.763416 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 215/433] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:42 2022] GET /identity/v3/projects/aa7a11e47f684403b0ebf54d869da5be => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:42.772300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253a201-7e35-4a7b-86bb-6f5e0822aa26 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:42.823324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 219/434] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:42.829387 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41daf87c-4d0a-45f7-9258-b95c9814ef02 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:42.849401 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:42.927911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253a201-7e35-4a7b-86bb-6f5e0822aa26 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:42.965129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41daf87c-4d0a-45f7-9258-b95c9814ef02 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:42.989056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10239d8a-5d23-4dca-b6d6-f41f27cf3391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:42.989930 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10239d8a-5d23-4dca-b6d6-f41f27cf3391 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:42.990214 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10239d8a-5d23-4dca-b6d6-f41f27cf3391 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:42.990489 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10239d8a-5d23-4dca-b6d6-f41f27cf3391 None admin] PATH_INFO: `/v3/groups/859c364715ee47fabc3a3027a878e3d9` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:42.992336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10239d8a-5d23-4dca-b6d6-f41f27cf3391 None admin] RBAC: Authorizing `identity:get_group(group_id=859c364715ee47fabc3a3027a878e3d9)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:42.993940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10239d8a-5d23-4dca-b6d6-f41f27cf3391 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:42.995899 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 216/435] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Jun 16 18:45:42 2022] GET /identity/v3/groups/859c364715ee47fabc3a3027a878e3d9 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:42.996915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b917346-2546-42d5-ac7e-71345d6c24a8 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:42.997213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b917346-2546-42d5-ac7e-71345d6c24a8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:42.997520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b917346-2546-42d5-ac7e-71345d6c24a8 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:43.006126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 217/436] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:42 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:43.007273 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b917346-2546-42d5-ac7e-71345d6c24a8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:43.012654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad8bbb-e194-47a3-89b0-df14f2cdf054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:43.013529 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad8bbb-e194-47a3-89b0-df14f2cdf054 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:43.013814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad8bbb-e194-47a3-89b0-df14f2cdf054 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:43.014097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad8bbb-e194-47a3-89b0-df14f2cdf054 None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:43.015335 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad8bbb-e194-47a3-89b0-df14f2cdf054 None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:43.016799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad8bbb-e194-47a3-89b0-df14f2cdf054 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:43.019691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 220/437] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:42 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:43.026146 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8873d4b-b912-412a-8731-6d71fb3cc7ac None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:43.026441 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8873d4b-b912-412a-8731-6d71fb3cc7ac None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:43.026772 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8873d4b-b912-412a-8731-6d71fb3cc7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:43.048732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e8873d4b-b912-412a-8731-6d71fb3cc7ac None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:43.050110 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e8873d4b-b912-412a-8731-6d71fb3cc7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:43.051078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:43.125094 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 221/438] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:43.125835 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8873d4b-b912-412a-8731-6d71fb3cc7ac None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:43.137185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:43.145870 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b917346-2546-42d5-ac7e-71345d6c24a8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:43.181170 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b724e-d18b-4e48-ac89-31bd36e6d770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:43.181882 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b724e-d18b-4e48-ac89-31bd36e6d770 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:43.182194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b724e-d18b-4e48-ac89-31bd36e6d770 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:43.182524 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b724e-d18b-4e48-ac89-31bd36e6d770 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:43.183383 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b724e-d18b-4e48-ac89-31bd36e6d770 None admin] RBAC: Adding query filter params (group.id=859c364715ee47fabc3a3027a878e3d9, role.id=662cb9b5dff94cdb83f292ad20d1d430, scope.project.id=aa7a11e47f684403b0ebf54d869da5be) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:43.183697 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b724e-d18b-4e48-ac89-31bd36e6d770 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:43.185128 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b724e-d18b-4e48-ac89-31bd36e6d770 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:43.191013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 218/439] 10.222.0.70 () {56 vars in 1376 bytes} [Thu Jun 16 18:45:43 2022] GET /identity/v3/role_assignments?group.id=859c364715ee47fabc3a3027a878e3d9&scope.project.id=aa7a11e47f684403b0ebf54d869da5be&role.id=662cb9b5dff94cdb83f292ad20d1d430 => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:43.192205 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25913768-004b-4662-92ca-a2f8b57eb64b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:43.192493 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25913768-004b-4662-92ca-a2f8b57eb64b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:43.192772 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25913768-004b-4662-92ca-a2f8b57eb64b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:43.194003 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 219/440] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:43.194791 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25913768-004b-4662-92ca-a2f8b57eb64b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:43.241993 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8873d4b-b912-412a-8731-6d71fb3cc7ac None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:43.283304 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8227e-e3a1-475e-9d33-1fbddf55db7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:43.284273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8227e-e3a1-475e-9d33-1fbddf55db7e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:43.284557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8227e-e3a1-475e-9d33-1fbddf55db7e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:43.284831 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8227e-e3a1-475e-9d33-1fbddf55db7e None admin] PATH_INFO: `/v3/groups/859c364715ee47fabc3a3027a878e3d9` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:43.286288 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8227e-e3a1-475e-9d33-1fbddf55db7e None admin] RBAC: Authorizing `identity:get_group(group_id=859c364715ee47fabc3a3027a878e3d9)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:43.287836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8227e-e3a1-475e-9d33-1fbddf55db7e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:43.290206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 222/441] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Jun 16 18:45:43 2022] GET /identity/v3/groups/859c364715ee47fabc3a3027a878e3d9 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:43.294988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c65df-acd6-44c9-839c-7ef34c060995 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:43.294988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c65df-acd6-44c9-839c-7ef34c060995 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:43.294988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c65df-acd6-44c9-839c-7ef34c060995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:43.314938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-a13c65df-acd6-44c9-839c-7ef34c060995 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:43.315714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:43.317537 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a13c65df-acd6-44c9-839c-7ef34c060995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:43.324563 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25913768-004b-4662-92ca-a2f8b57eb64b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:43.363184 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70278789-2c91-4a40-9825-2363848d8608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:43.364987 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70278789-2c91-4a40-9825-2363848d8608 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:43.365267 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70278789-2c91-4a40-9825-2363848d8608 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:43.365538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70278789-2c91-4a40-9825-2363848d8608 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:43.376719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70278789-2c91-4a40-9825-2363848d8608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:43.385323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-70278789-2c91-4a40-9825-2363848d8608 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:45:43.385323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:43.385323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:43.385323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:43.385323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:43.385323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:43.385323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:43.385323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:43.385323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:43.385323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:43.385323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:43.385323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:43.385323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:43.385323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:43.385323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:43.385323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:43.386240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:43.386240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:43.386240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:43.386240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:43.386240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:43.386240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:43.386240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:43.386240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:43.386240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:43.386240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:43.386240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:43.386240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:43.386240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:43.386240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:43.386240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:43.386240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:43.387077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:43.387077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:43.387077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:43.387077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:43.387077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:43.387077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:43.387077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:43.387077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:43.387077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:43.387077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:43.387077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:45:43.387077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:43.387077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 220/442] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:45:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:43.389844 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70278789-2c91-4a40-9825-2363848d8608 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:43.405168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 223/443] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:43.405889 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c65df-acd6-44c9-839c-7ef34c060995 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:43.420539 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:43.521127 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70278789-2c91-4a40-9825-2363848d8608 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:43.535158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c65df-acd6-44c9-839c-7ef34c060995 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:43.563365 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959d01a-102b-4bef-90ba-2b944ebf51ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:43.564090 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d01a-102b-4bef-90ba-2b944ebf51ed None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:43.564373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d01a-102b-4bef-90ba-2b944ebf51ed None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:43.564645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d01a-102b-4bef-90ba-2b944ebf51ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:43.565482 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959d01a-102b-4bef-90ba-2b944ebf51ed None admin] RBAC: Adding query filter params (group.id=859c364715ee47fabc3a3027a878e3d9, role.id=cf55e7c6398f45efaf10dec4fe153f8a, scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:43.565793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959d01a-102b-4bef-90ba-2b944ebf51ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:43.567244 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959d01a-102b-4bef-90ba-2b944ebf51ed None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:43.575345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 221/444] 10.222.0.70 () {56 vars in 1376 bytes} [Thu Jun 16 18:45:43 2022] GET /identity/v3/role_assignments?group.id=859c364715ee47fabc3a3027a878e3d9&scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df&role.id=cf55e7c6398f45efaf10dec4fe153f8a => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:43.576265 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959d01a-102b-4bef-90ba-2b944ebf51ed None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:43.595264 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc6059-78af-4c5e-89d5-ca5d5a5f42fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:43.595959 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc6059-78af-4c5e-89d5-ca5d5a5f42fa None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:43.596237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc6059-78af-4c5e-89d5-ca5d5a5f42fa None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:43.596508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc6059-78af-4c5e-89d5-ca5d5a5f42fa None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:43.597275 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc6059-78af-4c5e-89d5-ca5d5a5f42fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:43.597579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc6059-78af-4c5e-89d5-ca5d5a5f42fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:43.599023 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc6059-78af-4c5e-89d5-ca5d5a5f42fa None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:43.599638 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:43.613965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 224/445] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:43.615287 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc6059-78af-4c5e-89d5-ca5d5a5f42fa None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:43.696781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:43.707450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959d01a-102b-4bef-90ba-2b944ebf51ed None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:43.737437 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc6059-78af-4c5e-89d5-ca5d5a5f42fa None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:43.751054 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468b198-f085-4f6d-a724-b75dd00ad2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:43.751428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468b198-f085-4f6d-a724-b75dd00ad2ee None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:43.751700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468b198-f085-4f6d-a724-b75dd00ad2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:43.752606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468b198-f085-4f6d-a724-b75dd00ad2ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:43.764855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468b198-f085-4f6d-a724-b75dd00ad2ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:43.773692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b468b198-f085-4f6d-a724-b75dd00ad2ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:43.773692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:43.773692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:43.773692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:43.773692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:43.773692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:43.773692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:43.773692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:43.773692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:43.773692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:43.773692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:43.773692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:43.773692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:43.773692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:43.773692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:43.773692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:43.775337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:43.775337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:43.775337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:43.775337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:43.775337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:43.775337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:43.775337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:43.775337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:43.775337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:43.775337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:43.775337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:43.775337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:43.775337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:43.775337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:43.775337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:43.775337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:43.776299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:43.776299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:43.776299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:43.776299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:43.776299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:43.776299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:43.776299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:43.776299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:43.776299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:43.776299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:43.776299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:43.776299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:43.776299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 222/446] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:45:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:43.782126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468b198-f085-4f6d-a724-b75dd00ad2ee None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:43.783894 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f9d78-a661-4ef8-8647-75cbd36f4eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:43.784671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f9d78-a661-4ef8-8647-75cbd36f4eca None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:43.785058 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f9d78-a661-4ef8-8647-75cbd36f4eca None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:43.785677 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f9d78-a661-4ef8-8647-75cbd36f4eca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:43.876544 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f9d78-a661-4ef8-8647-75cbd36f4eca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:43.878428 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f9d78-a661-4ef8-8647-75cbd36f4eca None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:43.881067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:43.914083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468b198-f085-4f6d-a724-b75dd00ad2ee None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:43.957848 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d489ee-eb76-4467-a0da-c0e9a5e1fd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:43.957848 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d489ee-eb76-4467-a0da-c0e9a5e1fd5e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:43.957848 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d489ee-eb76-4467-a0da-c0e9a5e1fd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:43.957848 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d489ee-eb76-4467-a0da-c0e9a5e1fd5e None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:43.958273 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d489ee-eb76-4467-a0da-c0e9a5e1fd5e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:43.958273 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d489ee-eb76-4467-a0da-c0e9a5e1fd5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:43.959408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d489ee-eb76-4467-a0da-c0e9a5e1fd5e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:43.965483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:43.968767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 223/447] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:45:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:43.974149 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d489ee-eb76-4467-a0da-c0e9a5e1fd5e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:44.002512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 225/448] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:44.008054 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f9d78-a661-4ef8-8647-75cbd36f4eca None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:44.086170 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d489ee-eb76-4467-a0da-c0e9a5e1fd5e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:44.111168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f9d78-a661-4ef8-8647-75cbd36f4eca None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:44.120131 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa5715-60f9-4395-bc65-6d3f979651e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:44.120819 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa5715-60f9-4395-bc65-6d3f979651e7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:44.121124 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa5715-60f9-4395-bc65-6d3f979651e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:44.121407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa5715-60f9-4395-bc65-6d3f979651e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:44.131792 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:44.145721 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd592683-0ef6-4dc6-98cb-68e92feb3043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:44.146595 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd592683-0ef6-4dc6-98cb-68e92feb3043 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:44.146875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd592683-0ef6-4dc6-98cb-68e92feb3043 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:44.147152 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd592683-0ef6-4dc6-98cb-68e92feb3043 None admin] PATH_INFO: `/v3/projects/417f1f7349214e82b739864de0303f59` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:44.148330 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd592683-0ef6-4dc6-98cb-68e92feb3043 None admin] RBAC: Authorizing `identity:get_project(project_id=417f1f7349214e82b739864de0303f59)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:44.149828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd592683-0ef6-4dc6-98cb-68e92feb3043 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:44.151530 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 226/449] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:44 2022] GET /identity/v3/projects/417f1f7349214e82b739864de0303f59 => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:44.156842 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd592683-0ef6-4dc6-98cb-68e92feb3043 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:44.192744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa5715-60f9-4395-bc65-6d3f979651e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:44.194378 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa5715-60f9-4395-bc65-6d3f979651e7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:44.196353 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:44.259600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd592683-0ef6-4dc6-98cb-68e92feb3043 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:44.294159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ab008-a4dd-44e9-9cbe-459314895a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:44.295130 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ab008-a4dd-44e9-9cbe-459314895a9b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:44.295476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ab008-a4dd-44e9-9cbe-459314895a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:44.295636 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 224/450] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:45:44.295761 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ab008-a4dd-44e9-9cbe-459314895a9b None admin] PATH_INFO: `/v3/groups/578672aa776448f8a45473865665777c` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:44.296913 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ab008-a4dd-44e9-9cbe-459314895a9b None admin] RBAC: Authorizing `identity:get_group(group_id=578672aa776448f8a45473865665777c)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:44.298329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ab008-a4dd-44e9-9cbe-459314895a9b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:44.300069 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 227/451] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Jun 16 18:45:44 2022] GET /identity/v3/groups/578672aa776448f8a45473865665777c => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:44.301927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa5715-60f9-4395-bc65-6d3f979651e7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:44.306141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ab008-a4dd-44e9-9cbe-459314895a9b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:44.343232 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:44.401315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa5715-60f9-4395-bc65-6d3f979651e7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:44.404536 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ab008-a4dd-44e9-9cbe-459314895a9b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:44.411717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:44.434560 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5046d8e-dcd8-41c2-b00c-7d64db26b0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:44.435438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5046d8e-dcd8-41c2-b00c-7d64db26b0cb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:44.435717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5046d8e-dcd8-41c2-b00c-7d64db26b0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:44.435987 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5046d8e-dcd8-41c2-b00c-7d64db26b0cb None admin] PATH_INFO: `/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:44.437145 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5046d8e-dcd8-41c2-b00c-7d64db26b0cb None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b1c5f6c0e4bb4bf381d5aef7a66df)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:44.438600 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5046d8e-dcd8-41c2-b00c-7d64db26b0cb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:44.439143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b4316-011b-4537-8daf-7880721630ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:44.439794 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b4316-011b-4537-8daf-7880721630ca None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:44.440079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b4316-011b-4537-8daf-7880721630ca None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:44.440283 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 225/452] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:45:44 2022] GET /identity/v3/projects/5d6b1c5f6c0e4bb4bf381d5aef7a66df => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:44.440373 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b4316-011b-4537-8daf-7880721630ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:44.441327 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b4316-011b-4537-8daf-7880721630ca None admin] RBAC: Adding query filter params (group.id=578672aa776448f8a45473865665777c, role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=417f1f7349214e82b739864de0303f59) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:44.441629 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b4316-011b-4537-8daf-7880721630ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:44.443106 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b4316-011b-4537-8daf-7880721630ca None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:44.445572 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5046d8e-dcd8-41c2-b00c-7d64db26b0cb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:44.449366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 228/453] 10.222.0.70 () {56 vars in 1376 bytes} [Thu Jun 16 18:45:44 2022] GET /identity/v3/role_assignments?group.id=578672aa776448f8a45473865665777c&scope.project.id=417f1f7349214e82b739864de0303f59&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:44.552392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5046d8e-dcd8-41c2-b00c-7d64db26b0cb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:44.562714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:44.590014 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed658421-3804-4fb5-89e2-3055f8b1fbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:44.590914 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed658421-3804-4fb5-89e2-3055f8b1fbcc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:44.591202 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed658421-3804-4fb5-89e2-3055f8b1fbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:44.591478 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed658421-3804-4fb5-89e2-3055f8b1fbcc None admin] PATH_INFO: `/v3/groups/859c364715ee47fabc3a3027a878e3d9` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:44.592716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed658421-3804-4fb5-89e2-3055f8b1fbcc None admin] RBAC: Authorizing `identity:get_group(group_id=859c364715ee47fabc3a3027a878e3d9)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:44.594231 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed658421-3804-4fb5-89e2-3055f8b1fbcc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:44.596038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 226/454] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Jun 16 18:45:44 2022] GET /identity/v3/groups/859c364715ee47fabc3a3027a878e3d9 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:44.601908 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b4316-011b-4537-8daf-7880721630ca None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:44.713118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b4316-011b-4537-8daf-7880721630ca None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:44.745390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e91b66-3d1f-42f0-aee2-3f60f798536a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:44.746033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e91b66-3d1f-42f0-aee2-3f60f798536a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:44.746319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e91b66-3d1f-42f0-aee2-3f60f798536a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:44.746617 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e91b66-3d1f-42f0-aee2-3f60f798536a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:44.747438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e91b66-3d1f-42f0-aee2-3f60f798536a None admin] RBAC: Adding query filter params (group.id=859c364715ee47fabc3a3027a878e3d9, role.id=662cb9b5dff94cdb83f292ad20d1d430, scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:44.747752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e91b66-3d1f-42f0-aee2-3f60f798536a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:44.749187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e91b66-3d1f-42f0-aee2-3f60f798536a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:44.754933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 229/455] 10.222.0.70 () {56 vars in 1376 bytes} [Thu Jun 16 18:45:44 2022] GET /identity/v3/role_assignments?group.id=859c364715ee47fabc3a3027a878e3d9&scope.project.id=5d6b1c5f6c0e4bb4bf381d5aef7a66df&role.id=662cb9b5dff94cdb83f292ad20d1d430 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:47.392554 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d2f26-ccf0-4664-ab8e-b290297bea84 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:47.393648 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d2f26-ccf0-4664-ab8e-b290297bea84 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:47.394064 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d2f26-ccf0-4664-ab8e-b290297bea84 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:47.395815 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 227/456] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:47.401485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b35f3-93e4-4161-8a53-900992942c43 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:47.402234 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b35f3-93e4-4161-8a53-900992942c43 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:47.402681 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b35f3-93e4-4161-8a53-900992942c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:47.422542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-450b35f3-93e4-4161-8a53-900992942c43 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:47.424028 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-450b35f3-93e4-4161-8a53-900992942c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:47.468539 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:47.523398 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 230/457] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:47.559511 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d2f26-ccf0-4664-ab8e-b290297bea84 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:47.653723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc3f04-1964-4cfa-a7e0-db4b8a18564b None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:47.653928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc3f04-1964-4cfa-a7e0-db4b8a18564b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:47.654429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc3f04-1964-4cfa-a7e0-db4b8a18564b None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:47.656000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 231/458] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:45:47.661125 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0e43e-8495-4f83-a7fa-f4da7d752fb6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:47.661644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0e43e-8495-4f83-a7fa-f4da7d752fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:47.662105 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0e43e-8495-4f83-a7fa-f4da7d752fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:47.699653 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-9ec0e43e-8495-4f83-a7fa-f4da7d752fb6 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:47.701151 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9ec0e43e-8495-4f83-a7fa-f4da7d752fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:47.717445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d2f26-ccf0-4664-ab8e-b290297bea84 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:47.780370 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48539c2-bf06-447d-bc0a-3064810f3493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:47.781466 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48539c2-bf06-447d-bc0a-3064810f3493 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:47.781924 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48539c2-bf06-447d-bc0a-3064810f3493 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:47.782371 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48539c2-bf06-447d-bc0a-3064810f3493 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:47.799989 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48539c2-bf06-447d-bc0a-3064810f3493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:47.802759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48539c2-bf06-447d-bc0a-3064810f3493 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:47.828852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f48539c2-bf06-447d-bc0a-3064810f3493 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:47.828852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:47.828852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:47.828852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:47.828852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:47.828852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:47.828852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:47.828852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:47.828852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:47.828852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:47.828852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:47.828852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:47.828852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:45:47.828852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:45:47.828852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:45:47.829761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:45:47.829761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:47.829761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:47.829761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:47.829761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:47.829761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:47.829761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:47.829761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:47.829761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:47.829761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:47.829761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:47.829761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:47.829761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:47.829761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:47.829761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:47.829761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:47.829761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:47.830634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:45:47.830634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:45:47.830634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:47.830634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:47.832691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 232/459] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:47.834066 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 228/460] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:47.840480 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0e43e-8495-4f83-a7fa-f4da7d752fb6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:47.859658 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48539c2-bf06-447d-bc0a-3064810f3493 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:47.916327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:47.958049 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:47.967029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0e43e-8495-4f83-a7fa-f4da7d752fb6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:48.009339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c81b5-2b24-42a9-b1ef-690fa75c8ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:48.010339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c81b5-2b24-42a9-b1ef-690fa75c8ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:48.010826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c81b5-2b24-42a9-b1ef-690fa75c8ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:48.011280 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c81b5-2b24-42a9-b1ef-690fa75c8ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:48.012168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c81b5-2b24-42a9-b1ef-690fa75c8ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:48.012620 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c81b5-2b24-42a9-b1ef-690fa75c8ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:48.014182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c81b5-2b24-42a9-b1ef-690fa75c8ad8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:48.029865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 233/461] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:48.035011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c81b5-2b24-42a9-b1ef-690fa75c8ad8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:48.087089 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48539c2-bf06-447d-bc0a-3064810f3493 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:48.130230 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b42c43-5137-4922-9148-2dfd1fe547a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:48.131501 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b42c43-5137-4922-9148-2dfd1fe547a1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:48.132011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b42c43-5137-4922-9148-2dfd1fe547a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:48.132519 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b42c43-5137-4922-9148-2dfd1fe547a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:48.145573 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b42c43-5137-4922-9148-2dfd1fe547a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:48.147528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b42c43-5137-4922-9148-2dfd1fe547a1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:48.171004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-02b42c43-5137-4922-9148-2dfd1fe547a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:48.171004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:48.171004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:48.171004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:48.171004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:48.171004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:48.171004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:48.171004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:48.171004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:48.171004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:48.171004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:48.171004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:48.171004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:45:48.171004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:45:48.171004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:45:48.171925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:45:48.171925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:48.171925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:48.171925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:48.171925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:48.171925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:48.171925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:48.171925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:48.171925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:48.171925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:48.171925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:48.171925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:48.171925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:48.171925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:48.171925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:48.171925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:48.171925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:48.172800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:45:48.172800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:45:48.172800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:48.172800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:48.174780 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 229/462] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:45:48.176353 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3e1d6-7bae-4075-bdbe-03a7c199565e None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:48.176839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3e1d6-7bae-4075-bdbe-03a7c199565e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:48.177313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3e1d6-7bae-4075-bdbe-03a7c199565e None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:48.178148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c81b5-2b24-42a9-b1ef-690fa75c8ad8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:48.179319 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 230/463] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:48.181118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3e1d6-7bae-4075-bdbe-03a7c199565e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:48.222383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ddfc2-c964-41bb-a98e-f0f06828e7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:48.223098 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ddfc2-c964-41bb-a98e-f0f06828e7f8 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:48.223382 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ddfc2-c964-41bb-a98e-f0f06828e7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:48.223662 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ddfc2-c964-41bb-a98e-f0f06828e7f8 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:48.224367 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ddfc2-c964-41bb-a98e-f0f06828e7f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:48.225807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ddfc2-c964-41bb-a98e-f0f06828e7f8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:48.247899 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 234/464] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:45:48 2022] POST /identity/v3/users => generated 246 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:45:48.249173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82868bb-9113-4738-afa4-7abbf6bdb844 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:48.249517 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82868bb-9113-4738-afa4-7abbf6bdb844 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:48.249805 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82868bb-9113-4738-afa4-7abbf6bdb844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:48.272382 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d82868bb-9113-4738-afa4-7abbf6bdb844 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:48.274324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d82868bb-9113-4738-afa4-7abbf6bdb844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:48.279013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:48.296667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:48.327399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3e1d6-7bae-4075-bdbe-03a7c199565e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:48.370602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 235/465] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:48.370602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1460ec-e859-473b-803c-64f94195ad01 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:48.370602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1460ec-e859-473b-803c-64f94195ad01 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:48.370602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1460ec-e859-473b-803c-64f94195ad01 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:48.370602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 236/466] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:48.374012 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abd4f3-4135-4b9a-b146-a1220ce17f4e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:48.374301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abd4f3-4135-4b9a-b146-a1220ce17f4e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:48.374601 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abd4f3-4135-4b9a-b146-a1220ce17f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:48.390904 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459cc6b-ea08-4838-80f1-076ceb1220aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:48.391593 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459cc6b-ea08-4838-80f1-076ceb1220aa None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:48.391879 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459cc6b-ea08-4838-80f1-076ceb1220aa None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:48.392155 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459cc6b-ea08-4838-80f1-076ceb1220aa None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:48.392952 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459cc6b-ea08-4838-80f1-076ceb1220aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:48.393739 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459cc6b-ea08-4838-80f1-076ceb1220aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:48.397477 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459cc6b-ea08-4838-80f1-076ceb1220aa None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:48.405947 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-74abd4f3-4135-4b9a-b146-a1220ce17f4e None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:48.407382 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-74abd4f3-4135-4b9a-b146-a1220ce17f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:48.417797 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 231/467] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:48.419471 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459cc6b-ea08-4838-80f1-076ceb1220aa None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:48.498691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 237/468] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:48.499909 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abd4f3-4135-4b9a-b146-a1220ce17f4e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:48.549272 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459cc6b-ea08-4838-80f1-076ceb1220aa None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:48.583828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:48.589922 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206e7b5-c87d-4d0d-bf75-809b90aa7aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:48.592774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206e7b5-c87d-4d0d-bf75-809b90aa7aef None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:48.592774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206e7b5-c87d-4d0d-bf75-809b90aa7aef None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:48.592774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206e7b5-c87d-4d0d-bf75-809b90aa7aef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:48.600372 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:48.602757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206e7b5-c87d-4d0d-bf75-809b90aa7aef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:48.604329 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206e7b5-c87d-4d0d-bf75-809b90aa7aef None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:48.626979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b206e7b5-c87d-4d0d-bf75-809b90aa7aef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:48.626979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:48.626979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:48.626979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:48.626979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:48.626979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:48.626979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:48.626979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:48.626979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:48.626979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:48.626979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:48.626979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:48.626979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:45:48.626979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:45:48.626979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:45:48.628092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:45:48.628092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:48.628092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:48.628092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:48.628092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:48.628092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:48.628092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:48.628092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:48.628092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:48.628092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:48.628092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:48.628092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:48.628092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:48.628092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:48.628092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:48.628092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:48.628092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:48.628948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:45:48.628948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:45:48.628948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:48.628948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:48.628948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 232/469] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:48.629521 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206e7b5-c87d-4d0d-bf75-809b90aa7aef None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:48.632701 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abd4f3-4135-4b9a-b146-a1220ce17f4e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:48.676687 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559fa27-aa2d-49f1-9ee7-d562dbf2dc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:48.677366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559fa27-aa2d-49f1-9ee7-d562dbf2dc6c None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:48.678007 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559fa27-aa2d-49f1-9ee7-d562dbf2dc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:48.678289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559fa27-aa2d-49f1-9ee7-d562dbf2dc6c None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:48.679043 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559fa27-aa2d-49f1-9ee7-d562dbf2dc6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:48.681401 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559fa27-aa2d-49f1-9ee7-d562dbf2dc6c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:48.707577 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 238/470] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:45:48 2022] POST /identity/v3/users => generated 248 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:45:48.715432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559fa27-aa2d-49f1-9ee7-d562dbf2dc6c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:48.747435 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206e7b5-c87d-4d0d-bf75-809b90aa7aef None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:48.793221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca0c79-4c6f-4c3a-a615-5590aa3521a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:48.793969 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca0c79-4c6f-4c3a-a615-5590aa3521a4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:48.794259 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca0c79-4c6f-4c3a-a615-5590aa3521a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:48.794536 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca0c79-4c6f-4c3a-a615-5590aa3521a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:48.825411 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca0c79-4c6f-4c3a-a615-5590aa3521a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:48.827222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca0c79-4c6f-4c3a-a615-5590aa3521a4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:48.865545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b1ca0c79-4c6f-4c3a-a615-5590aa3521a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:48.865545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:48.865545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:48.865545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:48.865545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:48.865545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:48.865545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:48.865545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:48.865545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:48.865545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:48.865545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:48.865545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:48.865545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:45:48.865545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:45:48.865545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:45:48.867742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:45:48.867742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:48.867742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:48.867742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:48.867742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:48.867742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:48.867742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:48.867742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:48.867742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:48.867742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:48.867742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:48.867742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:48.867742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:48.867742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:48.867742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:48.867742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:48.867742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:48.869822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:45:48.869822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:45:48.869822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:48.869822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:48.871425 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 233/471] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:48.875021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca0c79-4c6f-4c3a-a615-5590aa3521a4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:48.934475 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559fa27-aa2d-49f1-9ee7-d562dbf2dc6c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:48.944027 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:48.966003 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:48.979641 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355d9ab-0a02-4ed1-ae12-3da016825483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:48.979839 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355d9ab-0a02-4ed1-ae12-3da016825483 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:48.980094 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355d9ab-0a02-4ed1-ae12-3da016825483 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:48.980340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355d9ab-0a02-4ed1-ae12-3da016825483 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:48.981087 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355d9ab-0a02-4ed1-ae12-3da016825483 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:48.981396 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355d9ab-0a02-4ed1-ae12-3da016825483 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:48.984535 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355d9ab-0a02-4ed1-ae12-3da016825483 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:49.002685 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 239/472] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:49.009168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355d9ab-0a02-4ed1-ae12-3da016825483 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:49.044260 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca0c79-4c6f-4c3a-a615-5590aa3521a4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:49.080516 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52bbd1a-0fbb-45f4-aee3-65dd491e3340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:49.081130 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52bbd1a-0fbb-45f4-aee3-65dd491e3340 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:49.081421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52bbd1a-0fbb-45f4-aee3-65dd491e3340 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:49.081700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52bbd1a-0fbb-45f4-aee3-65dd491e3340 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:49.082431 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52bbd1a-0fbb-45f4-aee3-65dd491e3340 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:49.082783 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52bbd1a-0fbb-45f4-aee3-65dd491e3340 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:49.084157 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52bbd1a-0fbb-45f4-aee3-65dd491e3340 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:49.097247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 234/473] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:49.103700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52bbd1a-0fbb-45f4-aee3-65dd491e3340 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:49.122994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355d9ab-0a02-4ed1-ae12-3da016825483 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:49.170939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2b77d-10fa-47ad-889a-c50f0843f5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:49.171656 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2b77d-10fa-47ad-889a-c50f0843f5f4 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:49.171937 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2b77d-10fa-47ad-889a-c50f0843f5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:49.172210 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2b77d-10fa-47ad-889a-c50f0843f5f4 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:49.172940 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2b77d-10fa-47ad-889a-c50f0843f5f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:49.174386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2b77d-10fa-47ad-889a-c50f0843f5f4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:49.206717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 240/474] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:45:49 2022] POST /identity/v3/users => generated 248 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:45:49.252011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52bbd1a-0fbb-45f4-aee3-65dd491e3340 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:49.255478 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:49.295382 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd6b40-f434-4902-ac62-95c1c7438e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:49.295991 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd6b40-f434-4902-ac62-95c1c7438e32 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:49.296275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd6b40-f434-4902-ac62-95c1c7438e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:49.296643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd6b40-f434-4902-ac62-95c1c7438e32 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:49.297369 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd6b40-f434-4902-ac62-95c1c7438e32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:49.298862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd6b40-f434-4902-ac62-95c1c7438e32 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:49.324329 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 235/475] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:45:49 2022] POST /identity/v3/users => generated 249 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:45:49.356760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413f130-d08f-41c6-aae9-7d0b68e5adc1 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:49.357398 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413f130-d08f-41c6-aae9-7d0b68e5adc1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:49.357879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413f130-d08f-41c6-aae9-7d0b68e5adc1 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:49.359648 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 241/476] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:49.365961 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c357e-f8bb-4554-bb2f-c8a36cc1f20f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:49.366273 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c357e-f8bb-4554-bb2f-c8a36cc1f20f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:49.366539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c357e-f8bb-4554-bb2f-c8a36cc1f20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:49.391291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ab6c357e-f8bb-4554-bb2f-c8a36cc1f20f None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:49.392546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ab6c357e-f8bb-4554-bb2f-c8a36cc1f20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:49.478140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 236/477] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:49.503649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413f130-d08f-41c6-aae9-7d0b68e5adc1 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:49.560798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:49.629274 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413f130-d08f-41c6-aae9-7d0b68e5adc1 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:49.667465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee242a-8061-4116-b686-904927988b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:49.668153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee242a-8061-4116-b686-904927988b12 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:49.668433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee242a-8061-4116-b686-904927988b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:49.668710 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee242a-8061-4116-b686-904927988b12 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:49.669453 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee242a-8061-4116-b686-904927988b12 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:49.670868 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee242a-8061-4116-b686-904927988b12 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:49.679328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.sql.core [None req-abee242a-8061-4116-b686-904927988b12 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 18:45:49.679328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 18:45:49.679328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [parameters: {'id': 'd9da31f98aa0452eac22dc3188737015', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 18:45:49.679328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80283) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 18:45:49.699814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-abee242a-8061-4116-b686-904927988b12 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 18:45:49.699814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:49.699814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:45:49.699814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:45:49.699814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:45:49.699814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:45:49.699814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:45:49.699814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:45:49.699814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:45:49.699814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:45:49.699814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:45:49.699814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:45:49.699814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:45:49.699814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application result.read() Jun 16 18:45:49.699814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return ref.to_dict() Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:45:49.700829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:45:49.701760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 18:45:49.701760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application yield resource Jun 16 18:45:49.701760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:45:49.701760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:45:49.701760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 18:45:49.701760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 18:45:49.701760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 18:45:49.701760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 18:45:49.701760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 18:45:49.701760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise self.value Jun 16 18:45:49.701760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 18:45:49.701760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 18:45:49.701760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 18:45:49.701760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application session.commit() Jun 16 18:45:49.701760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 18:45:49.701760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 18:45:49.701760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 18:45:49.702686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 18:45:49.702686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 18:45:49.702686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self.session.flush() Jun 16 18:45:49.702686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 18:45:49.702686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 18:45:49.702686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 18:45:49.702686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 18:45:49.702686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 18:45:49.702686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application compat.raise_( Jun 16 18:45:49.702686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:45:49.702686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception Jun 16 18:45:49.702686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 18:45:49.702686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 18:45:49.702686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 18:45:49.702686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 18:45:49.702686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 18:45:49.702686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 18:45:49.703602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 18:45:49.703602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 18:45:49.703602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 18:45:49.703602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 18:45:49.703602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 18:45:49.703602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 18:45:49.703602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 18:45:49.703602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 18:45:49.703602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 18:45:49.703602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 18:45:49.703602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 18:45:49.703602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 18:45:49.703602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 18:45:49.703602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 18:45:49.703602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:45:49.703602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception Jun 16 18:45:49.704427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:45:49.704427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:45:49.704427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:45:49.704427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:45:49.704427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:45:49.704427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:45:49.704427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:45:49.704427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:45:49.704427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:45:49.704427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:45:49.704427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:45:49.704427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application result.read() Jun 16 18:45:49.704427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:45:49.704427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:45:49.704427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:45:49.704427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:45:49.705230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:45:49.705230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:45:49.705230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:45:49.705230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:45:49.705230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 18:45:49.705230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 18:45:49.705230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [parameters: {'id': 'd9da31f98aa0452eac22dc3188737015', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 18:45:49.705230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 18:45:49.705230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:49.705230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:45:49.705230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:49.705230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:49.705230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:49.705230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:49.705230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:49.705230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:49.710461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:49.710461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:49.710461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:49.710461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:49.710461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:49.710461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:49.710461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 16 18:45:49.710461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 16 18:45:49.710461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:49.710461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:49.710461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 16 18:45:49.710461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 16 18:45:49.710461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 18:45:49.710461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 18:45:49.710461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 18:45:49.710461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:49.714182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 242/478] 10.222.0.70 () {60 vars in 1131 bytes} [Thu Jun 16 18:45:49 2022] POST /identity/v3/roles => generated 144 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:49.714360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c357e-f8bb-4554-bb2f-c8a36cc1f20f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:49.823570 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:49.833487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c357e-f8bb-4554-bb2f-c8a36cc1f20f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:49.872756 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f9c06-5991-4ef5-80be-4910dfea1015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:49.873580 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f9c06-5991-4ef5-80be-4910dfea1015 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:49.873863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f9c06-5991-4ef5-80be-4910dfea1015 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:49.874141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f9c06-5991-4ef5-80be-4910dfea1015 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:49.886706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f9c06-5991-4ef5-80be-4910dfea1015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:49.895268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-931f9c06-5991-4ef5-80be-4910dfea1015 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:49.895268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:49.895268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:49.895268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:49.895268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:49.895268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:49.895268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:49.895268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:49.895268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:49.895268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:49.895268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:49.895268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:49.895268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:49.895268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:49.895268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:49.895268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:49.896175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:49.896175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:49.896175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:49.896175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:49.896175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:49.896175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:49.896175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:49.896175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:49.896175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:49.896175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:49.896175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:49.896175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:49.896175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:49.896175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:49.896175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:49.896175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:49.897010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:49.897010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:49.897010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:49.897010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:49.897010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:49.897010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:49.897010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:49.897010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:49.897010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:49.897010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:49.897010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:45:49.897010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:49.897010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 237/479] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:45:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:49.903905 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee242a-8061-4116-b686-904927988b12 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:50.063379 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee242a-8061-4116-b686-904927988b12 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:50.077460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:50.113846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac016d-5420-4720-90e0-72009bd5fbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:50.114516 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac016d-5420-4720-90e0-72009bd5fbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:50.114826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac016d-5420-4720-90e0-72009bd5fbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:50.115106 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac016d-5420-4720-90e0-72009bd5fbb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:50.115856 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac016d-5420-4720-90e0-72009bd5fbb4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:50.116166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac016d-5420-4720-90e0-72009bd5fbb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:50.117707 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac016d-5420-4720-90e0-72009bd5fbb4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:50.129792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 243/480] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:45:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:51.194060 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c019f5-dfff-4b92-941a-a53de76fcae1 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:51.195093 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c019f5-dfff-4b92-941a-a53de76fcae1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:51.195662 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c019f5-dfff-4b92-941a-a53de76fcae1 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:51.197748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 238/481] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:51.203047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b49bb3-1a9f-4e9a-b786-f6e37e2a2394 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:51.203880 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b49bb3-1a9f-4e9a-b786-f6e37e2a2394 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:51.204349 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b49bb3-1a9f-4e9a-b786-f6e37e2a2394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:51.228088 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-09b49bb3-1a9f-4e9a-b786-f6e37e2a2394 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:51.229601 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-09b49bb3-1a9f-4e9a-b786-f6e37e2a2394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:51.318755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 244/482] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:51.343340 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c019f5-dfff-4b92-941a-a53de76fcae1 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:51.478268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded64ce-efd2-4e98-9464-27780075c1b4 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:51.478562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded64ce-efd2-4e98-9464-27780075c1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:51.478785 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded64ce-efd2-4e98-9464-27780075c1b4 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:51.480457 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 245/483] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:51.485793 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62c7c9-85ca-48bd-9561-ec2ecc3d4bc6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:51.486080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62c7c9-85ca-48bd-9561-ec2ecc3d4bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:51.486357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62c7c9-85ca-48bd-9561-ec2ecc3d4bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:51.500299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c019f5-dfff-4b92-941a-a53de76fcae1 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:51.504828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-9c62c7c9-85ca-48bd-9561-ec2ecc3d4bc6 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:51.506033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9c62c7c9-85ca-48bd-9561-ec2ecc3d4bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:51.569873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15904e-a113-4a14-bc21-ca091653976a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:51.571461 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15904e-a113-4a14-bc21-ca091653976a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:51.571925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15904e-a113-4a14-bc21-ca091653976a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:51.572211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15904e-a113-4a14-bc21-ca091653976a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:51.595915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15904e-a113-4a14-bc21-ca091653976a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:51.605578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-7d15904e-a113-4a14-bc21-ca091653976a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:45:51.605578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:51.605578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:51.605578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:51.605578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:51.605578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:51.605578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:51.605578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:51.605578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:51.605578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:51.605578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:51.605578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:51.605578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:51.605578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:51.605578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:51.605578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:51.606491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:51.606491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:51.606491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:51.606491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:51.606491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:51.606491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:51.606491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:51.606491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:51.606491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:51.606491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:51.606491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:51.606491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:51.606491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:51.606491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:51.606491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:51.606491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:51.607399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:51.607399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:51.607399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:51.607399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:51.607399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:51.607399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:51.607399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:51.607399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:51.607399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:51.607399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:51.607399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:45:51.607399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:51.610919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 239/484] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:45:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:51.613782 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15904e-a113-4a14-bc21-ca091653976a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:51.624248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 246/485] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:51.630950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:51.647985 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62c7c9-85ca-48bd-9561-ec2ecc3d4bc6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:51.723740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:51.733968 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15904e-a113-4a14-bc21-ca091653976a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:51.770949 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86b853-2f26-4259-be64-936b3946a4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:51.771632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86b853-2f26-4259-be64-936b3946a4ec None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:51.771921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86b853-2f26-4259-be64-936b3946a4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:51.772202 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86b853-2f26-4259-be64-936b3946a4ec None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:51.772960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86b853-2f26-4259-be64-936b3946a4ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:51.773268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86b853-2f26-4259-be64-936b3946a4ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:51.775729 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86b853-2f26-4259-be64-936b3946a4ec None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:51.784990 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 240/486] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:45:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:51.791462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86b853-2f26-4259-be64-936b3946a4ec None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:51.889797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62c7c9-85ca-48bd-9561-ec2ecc3d4bc6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:51.928058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:51.943611 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf446fc6-c148-4731-a956-7cf011674f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:51.947186 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf446fc6-c148-4731-a956-7cf011674f05 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:51.947186 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf446fc6-c148-4731-a956-7cf011674f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:51.947186 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf446fc6-c148-4731-a956-7cf011674f05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:51.956724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf446fc6-c148-4731-a956-7cf011674f05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:51.962229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86b853-2f26-4259-be64-936b3946a4ec None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:51.965942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-cf446fc6-c148-4731-a956-7cf011674f05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:45:51.965942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:51.965942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:51.965942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:51.965942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:51.965942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:51.965942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:51.965942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:51.965942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:51.965942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:51.965942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:51.965942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:51.965942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:51.965942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:51.965942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:51.965942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:51.966893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:51.966893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:51.966893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:51.966893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:51.966893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:51.966893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:51.966893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:51.966893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:51.966893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:51.966893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:51.966893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:51.966893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:51.966893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:51.966893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:51.966893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:51.966893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:51.967688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:51.967688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:51.967688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:51.967688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:51.967688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:51.967688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:51.967688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:51.967688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:51.967688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:51.967688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:51.967688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:45:51.967688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:51.969799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 247/487] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:45:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:45:51.971175 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd80bb-4783-43a4-9e73-94c1b7326800 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:51.971601 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd80bb-4783-43a4-9e73-94c1b7326800 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:51.972119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd80bb-4783-43a4-9e73-94c1b7326800 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:51.973546 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 248/488] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:51.978788 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd80bb-4783-43a4-9e73-94c1b7326800 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:52.011123 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e86f3-26c7-45be-bb35-08be5d4e9736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:52.012245 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e86f3-26c7-45be-bb35-08be5d4e9736 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:52.012685 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e86f3-26c7-45be-bb35-08be5d4e9736 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:52.013114 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e86f3-26c7-45be-bb35-08be5d4e9736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:52.027076 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e86f3-26c7-45be-bb35-08be5d4e9736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:52.028794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e86f3-26c7-45be-bb35-08be5d4e9736 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:52.047049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b93e86f3-26c7-45be-bb35-08be5d4e9736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:52.047049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:52.047049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:52.047049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:52.047049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:52.047049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:52.047049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:52.047049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:52.047049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:52.047049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:52.047049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:52.047049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:52.047049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:45:52.047049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:45:52.047049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:45:52.048164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:45:52.048164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:52.048164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:52.048164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:52.048164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:52.048164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:52.048164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:52.048164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:52.048164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:52.048164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:52.048164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:52.048164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:52.048164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:52.048164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:52.048164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:52.048164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:52.048164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:52.049211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:45:52.049211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:45:52.049211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:52.049211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:52.050797 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 241/489] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:45:52.052155 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9bd81-261a-4833-a5b4-6e21a16e7759 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:52.052597 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9bd81-261a-4833-a5b4-6e21a16e7759 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:52.053019 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9bd81-261a-4833-a5b4-6e21a16e7759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:52.071012 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a1b9bd81-261a-4833-a5b4-6e21a16e7759 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:52.072551 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a1b9bd81-261a-4833-a5b4-6e21a16e7759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:52.083285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:52.098987 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd80bb-4783-43a4-9e73-94c1b7326800 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:52.135314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbac63-2a6c-4f1c-ae9b-ada9629b4dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:52.135985 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbac63-2a6c-4f1c-ae9b-ada9629b4dfd None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:52.136271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbac63-2a6c-4f1c-ae9b-ada9629b4dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:52.136766 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbac63-2a6c-4f1c-ae9b-ada9629b4dfd None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:52.137292 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bbac63-2a6c-4f1c-ae9b-ada9629b4dfd None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:52.137602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bbac63-2a6c-4f1c-ae9b-ada9629b4dfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:52.139012 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bbac63-2a6c-4f1c-ae9b-ada9629b4dfd None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:52.144782 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 242/490] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:52.145463 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9bd81-261a-4833-a5b4-6e21a16e7759 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:52.154190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 249/491] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:45:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:52.155823 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361dea3-188c-490d-b2eb-6942eafc17c6 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:52.156305 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361dea3-188c-490d-b2eb-6942eafc17c6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:52.156833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361dea3-188c-490d-b2eb-6942eafc17c6 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:52.163033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 250/492] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:52.163832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361dea3-188c-490d-b2eb-6942eafc17c6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:52.214552 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:52.263723 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9bd81-261a-4833-a5b4-6e21a16e7759 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:52.281324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361dea3-188c-490d-b2eb-6942eafc17c6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:52.299778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bfd67-fde1-490d-b1d8-abc0061be721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:52.300437 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bfd67-fde1-490d-b1d8-abc0061be721 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:52.300715 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bfd67-fde1-490d-b1d8-abc0061be721 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:52.300986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bfd67-fde1-490d-b1d8-abc0061be721 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:52.301694 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bfd67-fde1-490d-b1d8-abc0061be721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:52.302006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bfd67-fde1-490d-b1d8-abc0061be721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:52.303386 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bfd67-fde1-490d-b1d8-abc0061be721 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:52.320211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 243/493] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:52.320211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512df3cf-7173-4efe-ad67-8f1938d7cc3a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:52.320211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512df3cf-7173-4efe-ad67-8f1938d7cc3a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:52.320211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512df3cf-7173-4efe-ad67-8f1938d7cc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:52.324604 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d7eb0-6736-4fff-a76d-3fe0757a95e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:52.326258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d7eb0-6736-4fff-a76d-3fe0757a95e0 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:52.326258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d7eb0-6736-4fff-a76d-3fe0757a95e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:52.326258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d7eb0-6736-4fff-a76d-3fe0757a95e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:52.336928 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-512df3cf-7173-4efe-ad67-8f1938d7cc3a None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:52.338183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-512df3cf-7173-4efe-ad67-8f1938d7cc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:52.341601 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d7eb0-6736-4fff-a76d-3fe0757a95e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:52.343072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d7eb0-6736-4fff-a76d-3fe0757a95e0 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:52.363440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-de3d7eb0-6736-4fff-a76d-3fe0757a95e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:52.363440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:52.363440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:52.363440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:52.363440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:52.363440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:52.363440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:52.363440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:52.363440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:52.363440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:52.363440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:52.363440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:52.363440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:45:52.363440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:45:52.363440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:45:52.364499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:45:52.364499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:52.364499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:52.364499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:52.364499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:52.364499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:52.364499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:52.364499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:52.364499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:52.364499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:52.364499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:52.364499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:52.364499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:52.364499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:52.364499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:52.364499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:52.364499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:52.365558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:45:52.365558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:45:52.365558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:52.365558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:52.367297 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 251/494] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:52.368294 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d7eb0-6736-4fff-a76d-3fe0757a95e0 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:52.371149 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:52.426708 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 244/495] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:52.427671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512df3cf-7173-4efe-ad67-8f1938d7cc3a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:52.482934 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:52.483783 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d7eb0-6736-4fff-a76d-3fe0757a95e0 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:52.521883 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d68b1-a2ea-4526-a176-70f0606f3595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:52.522617 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d68b1-a2ea-4526-a176-70f0606f3595 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:52.522910 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d68b1-a2ea-4526-a176-70f0606f3595 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:52.523191 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d68b1-a2ea-4526-a176-70f0606f3595 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:52.534403 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d68b1-a2ea-4526-a176-70f0606f3595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:52.542501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-204d68b1-a2ea-4526-a176-70f0606f3595 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:45:52.542501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:52.542501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:52.542501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:52.542501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:52.542501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:52.542501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:52.542501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:52.542501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:52.542501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:52.542501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:52.542501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:52.542501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:52.542501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:52.542501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:52.542501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:52.543488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:52.543488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:52.543488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:52.543488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:52.543488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:52.543488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:52.543488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:52.543488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:52.543488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:52.543488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:52.543488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:52.543488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:52.543488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:52.543488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:52.543488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:52.543488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:52.544300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:52.544300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:52.544300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:52.544300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:52.544300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:52.544300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:52.544300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:52.544300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:52.544300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:52.544300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:52.544300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:45:52.544300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:52.544300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 252/496] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:45:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:52.544997 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d68b1-a2ea-4526-a176-70f0606f3595 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:52.550891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512df3cf-7173-4efe-ad67-8f1938d7cc3a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:52.588989 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f55a6-f45a-4105-b501-3d2ab3fcf4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:52.590278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f55a6-f45a-4105-b501-3d2ab3fcf4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:52.590707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f55a6-f45a-4105-b501-3d2ab3fcf4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:52.590987 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f55a6-f45a-4105-b501-3d2ab3fcf4b3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:52.599222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f55a6-f45a-4105-b501-3d2ab3fcf4b3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:52.600903 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f55a6-f45a-4105-b501-3d2ab3fcf4b3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:52.611208 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:52.619753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-d74f55a6-f45a-4105-b501-3d2ab3fcf4b3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:45:52.619753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:52.619753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:52.619753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:52.619753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:52.619753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:52.619753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:52.619753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:52.619753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:52.619753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:52.619753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:52.619753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:52.619753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:45:52.619753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:45:52.619753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:45:52.619753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:52.620680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:52.621574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:52.621574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:52.621574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:45:52.621574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:45:52.621574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:45:52.621574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:45:52.621574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:45:52.621574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:45:52.621574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:45:52.621574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:52.621574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 245/497] 10.222.0.70 () {56 vars in 1093 bytes} [Thu Jun 16 18:45:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:52.622129 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f55a6-f45a-4105-b501-3d2ab3fcf4b3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:52.657689 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d68b1-a2ea-4526-a176-70f0606f3595 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:52.694318 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a13fbc-3d18-4db3-95ea-aa5fc20fae9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:52.695013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a13fbc-3d18-4db3-95ea-aa5fc20fae9c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:52.695442 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a13fbc-3d18-4db3-95ea-aa5fc20fae9c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:52.695898 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a13fbc-3d18-4db3-95ea-aa5fc20fae9c None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:52.696638 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a13fbc-3d18-4db3-95ea-aa5fc20fae9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:52.696949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a13fbc-3d18-4db3-95ea-aa5fc20fae9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:52.698340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a13fbc-3d18-4db3-95ea-aa5fc20fae9c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:52.715306 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 253/498] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:52.715306 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a13fbc-3d18-4db3-95ea-aa5fc20fae9c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:52.732804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f55a6-f45a-4105-b501-3d2ab3fcf4b3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:52.735188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:52.772755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c8ff7-6964-4a07-85ef-c2d4e78a5069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:52.774050 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c8ff7-6964-4a07-85ef-c2d4e78a5069 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:52.774358 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c8ff7-6964-4a07-85ef-c2d4e78a5069 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:52.774689 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c8ff7-6964-4a07-85ef-c2d4e78a5069 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:52.785514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c8ff7-6964-4a07-85ef-c2d4e78a5069 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:52.793711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-dd2c8ff7-6964-4a07-85ef-c2d4e78a5069 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:45:52.793711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:52.793711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:52.793711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:52.793711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:52.793711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:52.793711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:52.793711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:52.793711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:52.793711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:52.793711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:52.793711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:52.793711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:52.793711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:52.793711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:52.793711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:52.794651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:52.794651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:52.794651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:52.794651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:52.794651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:52.794651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:52.794651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:52.794651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:52.794651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:52.794651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:52.794651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:52.794651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:52.794651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:52.794651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:52.794651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:52.794651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:52.800395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:52.800395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:52.800395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:52.800395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:52.800395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:52.800395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:52.800395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:52.800395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:52.800395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:52.800395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:52.800395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:45:52.800395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:52.800395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 246/499] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:45:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:52.800395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c8ff7-6964-4a07-85ef-c2d4e78a5069 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:52.828333 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a13fbc-3d18-4db3-95ea-aa5fc20fae9c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:52.852536 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:52.865299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fb9b0-b290-45d8-b537-b17eb0ee9daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:52.866174 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fb9b0-b290-45d8-b537-b17eb0ee9daa None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:52.866619 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fb9b0-b290-45d8-b537-b17eb0ee9daa None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:52.867068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fb9b0-b290-45d8-b537-b17eb0ee9daa None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:52.868180 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fb9b0-b290-45d8-b537-b17eb0ee9daa None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:52.868662 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fb9b0-b290-45d8-b537-b17eb0ee9daa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:52.870173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fb9b0-b290-45d8-b537-b17eb0ee9daa None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:52.879390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 254/500] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:45:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:52.880352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fb9b0-b290-45d8-b537-b17eb0ee9daa None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:52.907541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c8ff7-6964-4a07-85ef-c2d4e78a5069 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:52.951075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ff0e0-ea33-4900-a6af-7ce651dfb70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:52.951762 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ff0e0-ea33-4900-a6af-7ce651dfb70b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:52.952054 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ff0e0-ea33-4900-a6af-7ce651dfb70b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:52.952336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ff0e0-ea33-4900-a6af-7ce651dfb70b None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:52.953180 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ff0e0-ea33-4900-a6af-7ce651dfb70b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:52.953517 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ff0e0-ea33-4900-a6af-7ce651dfb70b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:52.955041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ff0e0-ea33-4900-a6af-7ce651dfb70b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:52.993444 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 247/501] 10.222.0.70 () {56 vars in 1133 bytes} [Thu Jun 16 18:45:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:52.994256 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ff0e0-ea33-4900-a6af-7ce651dfb70b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:53.010324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fb9b0-b290-45d8-b537-b17eb0ee9daa None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:53.022196 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:53.052921 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fc5ae-2cc1-4525-bc17-32626e959eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:53.054215 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fc5ae-2cc1-4525-bc17-32626e959eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:53.054796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fc5ae-2cc1-4525-bc17-32626e959eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:53.055287 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fc5ae-2cc1-4525-bc17-32626e959eb5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:53.063954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fc5ae-2cc1-4525-bc17-32626e959eb5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:53.065489 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fc5ae-2cc1-4525-bc17-32626e959eb5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:53.083158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-188fc5ae-2cc1-4525-bc17-32626e959eb5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:45:53.083158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:53.083158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:53.083158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:53.083158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:53.083158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:53.083158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:53.083158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:53.083158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:53.083158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:53.083158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:53.083158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:53.083158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:45:53.083158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:45:53.083158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:45:53.083158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:53.084097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:53.085068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:53.085068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:53.085068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:45:53.085068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:45:53.085068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:45:53.085068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:45:53.085068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:45:53.085068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:45:53.085068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:45:53.085068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:53.085068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 255/502] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:53.085638 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fc5ae-2cc1-4525-bc17-32626e959eb5 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:53.110202 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ff0e0-ea33-4900-a6af-7ce651dfb70b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:53.131422 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:53.145471 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7099dfa-cc26-4005-b816-8dfe14b546b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:53.146138 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7099dfa-cc26-4005-b816-8dfe14b546b5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:53.146420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7099dfa-cc26-4005-b816-8dfe14b546b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:53.146724 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7099dfa-cc26-4005-b816-8dfe14b546b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:53.147464 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7099dfa-cc26-4005-b816-8dfe14b546b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:53.147766 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7099dfa-cc26-4005-b816-8dfe14b546b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:53.149438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7099dfa-cc26-4005-b816-8dfe14b546b5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:53.157395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 248/503] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:45:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:53.158180 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7099dfa-cc26-4005-b816-8dfe14b546b5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:53.190284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fc5ae-2cc1-4525-bc17-32626e959eb5 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:53.223660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ee20c-399d-4e72-a2ae-f9685e081b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:53.224435 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee20c-399d-4e72-a2ae-f9685e081b1b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:53.224713 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee20c-399d-4e72-a2ae-f9685e081b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:53.224982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee20c-399d-4e72-a2ae-f9685e081b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:53.235472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ee20c-399d-4e72-a2ae-f9685e081b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:53.236924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ee20c-399d-4e72-a2ae-f9685e081b1b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:53.245775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:53.254799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-d94ee20c-399d-4e72-a2ae-f9685e081b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:53.254799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:53.254799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:53.254799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:53.254799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:53.254799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:53.254799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:53.254799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:53.254799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:53.254799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:53.254799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:53.254799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:53.254799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:45:53.254799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:45:53.254799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:45:53.255664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:45:53.255664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:53.255664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:53.255664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:53.255664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:53.255664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:53.255664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:53.255664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:53.255664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:53.255664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:53.255664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:53.255664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:53.255664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:53.255664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:53.255664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:53.255664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:53.255664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:53.256508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:45:53.256508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:45:53.256508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:53.256508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:53.256508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 256/504] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:53.257233 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381d61e-1712-4bc6-bf80-a2191846c00e None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:53.257484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381d61e-1712-4bc6-bf80-a2191846c00e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:53.257814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381d61e-1712-4bc6-bf80-a2191846c00e None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:53.259280 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 257/505] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:53.259973 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381d61e-1712-4bc6-bf80-a2191846c00e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:53.267392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7099dfa-cc26-4005-b816-8dfe14b546b5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:53.300502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f6781-73e6-4172-96d9-6bc28de12c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:53.301139 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f6781-73e6-4172-96d9-6bc28de12c3f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:53.301418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f6781-73e6-4172-96d9-6bc28de12c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:53.301686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f6781-73e6-4172-96d9-6bc28de12c3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:53.358775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:53.366040 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381d61e-1712-4bc6-bf80-a2191846c00e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:53.371054 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f6781-73e6-4172-96d9-6bc28de12c3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:53.372514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f6781-73e6-4172-96d9-6bc28de12c3f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:53.400649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7e9dc-c735-43d2-be04-5c7ad562dde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:53.401375 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7e9dc-c735-43d2-be04-5c7ad562dde4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:53.401666 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7e9dc-c735-43d2-be04-5c7ad562dde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:53.401940 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7e9dc-c735-43d2-be04-5c7ad562dde4 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:53.402811 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7e9dc-c735-43d2-be04-5c7ad562dde4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:53.403123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7e9dc-c735-43d2-be04-5c7ad562dde4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:53.404565 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7e9dc-c735-43d2-be04-5c7ad562dde4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:53.441411 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 258/506] 10.222.0.70 () {56 vars in 1137 bytes} [Thu Jun 16 18:45:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:53.442108 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7e9dc-c735-43d2-be04-5c7ad562dde4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:53.455419 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:53.472280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 249/507] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:53.472280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f6781-73e6-4172-96d9-6bc28de12c3f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:53.541443 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7e9dc-c735-43d2-be04-5c7ad562dde4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:53.569978 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f6781-73e6-4172-96d9-6bc28de12c3f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:53.574718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ef267-4111-41f5-a4e8-f3e97a1cfdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:53.575503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ef267-4111-41f5-a4e8-f3e97a1cfdd1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:53.575784 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ef267-4111-41f5-a4e8-f3e97a1cfdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:53.576055 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ef267-4111-41f5-a4e8-f3e97a1cfdd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:53.586032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ef267-4111-41f5-a4e8-f3e97a1cfdd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:53.587592 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ef267-4111-41f5-a4e8-f3e97a1cfdd1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:53.605342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-0b6ef267-4111-41f5-a4e8-f3e97a1cfdd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:53.605342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:53.605342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:53.605342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:53.605342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:53.605342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:53.605342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:53.605342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:53.605342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:53.605342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:53.605342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:53.605342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:53.605342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:45:53.605342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:45:53.605342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:45:53.606227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:45:53.606227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:53.606227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:53.606227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:53.606227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:53.606227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:53.606227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:53.606227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:53.606227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:53.606227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:53.606227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:53.606227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:53.606227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:53.606227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:53.606227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:53.606227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:53.606227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:53.607115 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:45:53.607115 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:45:53.607115 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:53.607115 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:53.607338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4745c-c570-4340-bbf8-2ab7209aca1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:53.607424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 259/508] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:53.607491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4745c-c570-4340-bbf8-2ab7209aca1e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:53.607491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4745c-c570-4340-bbf8-2ab7209aca1e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:53.607491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4745c-c570-4340-bbf8-2ab7209aca1e None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:53.608123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34369b6c-7260-4ab6-b938-025c2a05f0a0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:53.608123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34369b6c-7260-4ab6-b938-025c2a05f0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:53.608274 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde4745c-c570-4340-bbf8-2ab7209aca1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:53.608341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34369b6c-7260-4ab6-b938-025c2a05f0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:53.608417 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde4745c-c570-4340-bbf8-2ab7209aca1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:53.609810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde4745c-c570-4340-bbf8-2ab7209aca1e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:53.619203 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:53.621621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 250/509] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:53.622425 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4745c-c570-4340-bbf8-2ab7209aca1e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:53.625252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-34369b6c-7260-4ab6-b938-025c2a05f0a0 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:53.627002 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-34369b6c-7260-4ab6-b938-025c2a05f0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:53.668067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:53.694516 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 260/510] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:53.695967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34369b6c-7260-4ab6-b938-025c2a05f0a0 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:53.732822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4745c-c570-4340-bbf8-2ab7209aca1e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:53.766247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01265554-cce2-4728-889b-d61e607d2546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:53.766919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01265554-cce2-4728-889b-d61e607d2546 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:53.767199 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01265554-cce2-4728-889b-d61e607d2546 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:53.767470 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01265554-cce2-4728-889b-d61e607d2546 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:53.803275 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34369b6c-7260-4ab6-b938-025c2a05f0a0 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:53.836710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01265554-cce2-4728-889b-d61e607d2546 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:53.838180 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01265554-cce2-4728-889b-d61e607d2546 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:53.840468 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba69a86-2497-48bb-b08a-325f52401bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:53.841285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba69a86-2497-48bb-b08a-325f52401bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:53.841564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba69a86-2497-48bb-b08a-325f52401bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:53.841830 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba69a86-2497-48bb-b08a-325f52401bc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:53.844571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:53.856567 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba69a86-2497-48bb-b08a-325f52401bc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:53.856567 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba69a86-2497-48bb-b08a-325f52401bc1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:53.873032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-eba69a86-2497-48bb-b08a-325f52401bc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:53.873032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:53.873032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:53.873032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:53.873032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:53.873032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:53.873032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:53.873032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:53.873032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:53.873032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:53.873032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:53.873032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:53.873032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:45:53.873032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:45:53.873032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:45:53.874421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:45:53.874421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:53.874421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:53.874421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:53.874421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:53.874421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:53.874421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:53.874421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:53.874421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:53.874421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:53.874421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:53.874421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:53.874421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:53.874421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:53.874421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:53.874421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:53.874421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:53.875795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:45:53.875795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:45:53.875795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:53.875795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:53.875795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 261/511] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:53.875795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba69a86-2497-48bb-b08a-325f52401bc1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:53.891067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:53.949011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 251/512] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:53.949858 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01265554-cce2-4728-889b-d61e607d2546 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:53.991935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba69a86-2497-48bb-b08a-325f52401bc1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:54.027383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c5633-9b7a-412f-9851-0bb8b727a504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:54.028040 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c5633-9b7a-412f-9851-0bb8b727a504 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:54.028318 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c5633-9b7a-412f-9851-0bb8b727a504 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:54.028590 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c5633-9b7a-412f-9851-0bb8b727a504 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:54.029298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c5633-9b7a-412f-9851-0bb8b727a504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:54.029599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c5633-9b7a-412f-9851-0bb8b727a504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:54.030997 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c5633-9b7a-412f-9851-0bb8b727a504 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:54.042327 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 262/513] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:54.043304 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c5633-9b7a-412f-9851-0bb8b727a504 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:54.052326 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01265554-cce2-4728-889b-d61e607d2546 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:54.087719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b68e2-f6fa-4b14-8b60-6e9950fccc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:54.088506 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b68e2-f6fa-4b14-8b60-6e9950fccc58 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:54.088780 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b68e2-f6fa-4b14-8b60-6e9950fccc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:54.089052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b68e2-f6fa-4b14-8b60-6e9950fccc58 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:54.091531 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:54.096394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b68e2-f6fa-4b14-8b60-6e9950fccc58 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:54.097918 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b68e2-f6fa-4b14-8b60-6e9950fccc58 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:54.112168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-0a8b68e2-f6fa-4b14-8b60-6e9950fccc58 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:45:54.112168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:54.112168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:54.112168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:54.112168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:54.112168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:54.112168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:54.112168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:54.112168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:54.112168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:54.112168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:54.112168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:54.112168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:45:54.112168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:45:54.112168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:45:54.112168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:54.113079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:54.113994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:54.113994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:54.113994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:45:54.113994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:45:54.113994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:45:54.113994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:45:54.113994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:45:54.113994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:45:54.113994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:45:54.113994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:54.113994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 252/514] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:45:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:54.114626 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b68e2-f6fa-4b14-8b60-6e9950fccc58 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:54.118046 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:54.145902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c5633-9b7a-412f-9851-0bb8b727a504 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:54.179263 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba8bd9-9d78-42e9-b9b2-4183bb46b7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:54.180153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba8bd9-9d78-42e9-b9b2-4183bb46b7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:54.180426 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba8bd9-9d78-42e9-b9b2-4183bb46b7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:54.180696 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba8bd9-9d78-42e9-b9b2-4183bb46b7b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:54.190936 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba8bd9-9d78-42e9-b9b2-4183bb46b7b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:54.192364 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba8bd9-9d78-42e9-b9b2-4183bb46b7b9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:54.209231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-42ba8bd9-9d78-42e9-b9b2-4183bb46b7b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:54.209231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:54.209231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:54.209231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:54.209231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:54.209231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:54.209231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:54.209231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:54.209231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:54.209231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:54.209231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:54.209231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:54.209231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:45:54.209231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:45:54.209231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:45:54.210095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:45:54.210095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:54.210095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:54.210095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:54.210095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:54.210095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:54.210095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:54.210095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:54.210095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:54.210095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:54.210095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:54.210095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:54.210095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:54.210095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:54.210095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:54.210095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:54.210095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:54.211609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:45:54.211609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:45:54.211609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:54.211609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:54.211609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 263/515] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:54.211920 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba8bd9-9d78-42e9-b9b2-4183bb46b7b9 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:54.219758 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b68e2-f6fa-4b14-8b60-6e9950fccc58 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:54.256193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c147d-63b8-4df8-9e2c-7172c39e2352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:54.256852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c147d-63b8-4df8-9e2c-7172c39e2352 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:54.257129 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c147d-63b8-4df8-9e2c-7172c39e2352 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:54.257405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c147d-63b8-4df8-9e2c-7172c39e2352 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:54.258323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c147d-63b8-4df8-9e2c-7172c39e2352 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:54.258659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c147d-63b8-4df8-9e2c-7172c39e2352 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:54.260043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c147d-63b8-4df8-9e2c-7172c39e2352 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:54.271167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 253/516] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:54.271939 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c147d-63b8-4df8-9e2c-7172c39e2352 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:54.311639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:54.315412 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba8bd9-9d78-42e9-b9b2-4183bb46b7b9 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:54.334622 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:54.352873 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0920a99-4d88-4339-863c-8580585bb5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:54.353669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0920a99-4d88-4339-863c-8580585bb5dd None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:54.353950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0920a99-4d88-4339-863c-8580585bb5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:54.354228 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0920a99-4d88-4339-863c-8580585bb5dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:54.365314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0920a99-4d88-4339-863c-8580585bb5dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:54.366807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0920a99-4d88-4339-863c-8580585bb5dd None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:54.376864 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c147d-63b8-4df8-9e2c-7172c39e2352 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:54.384644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-d0920a99-4d88-4339-863c-8580585bb5dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:54.384644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:54.384644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:54.384644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:54.384644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:54.384644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:54.384644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:54.384644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:54.384644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:54.384644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:54.384644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:54.384644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:54.384644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:45:54.384644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:45:54.384644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:45:54.385509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:45:54.385509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:54.385509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:54.385509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:54.385509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:54.385509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:54.385509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:54.385509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:54.385509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:54.385509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:54.385509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:54.385509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:54.385509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:54.385509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:54.385509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:54.385509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:54.385509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:54.386381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:45:54.386381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:45:54.386381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:54.386381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:54.386381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 264/517] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:54.386690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0920a99-4d88-4339-863c-8580585bb5dd None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:54.415853 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac277d8-922f-42c8-9ad4-435d89301d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:54.416567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac277d8-922f-42c8-9ad4-435d89301d66 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:54.416847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac277d8-922f-42c8-9ad4-435d89301d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:54.417123 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac277d8-922f-42c8-9ad4-435d89301d66 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:54.424493 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac277d8-922f-42c8-9ad4-435d89301d66 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:54.425915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac277d8-922f-42c8-9ad4-435d89301d66 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:54.439889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-4ac277d8-922f-42c8-9ad4-435d89301d66 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:45:54.439889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:54.439889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:54.439889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:54.439889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:54.439889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:54.439889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:54.439889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:54.439889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:54.439889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:54.439889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:54.439889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:54.439889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:45:54.439889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:45:54.439889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:45:54.439889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:54.440862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:54.441764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:54.441764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:54.441764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:45:54.441764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:45:54.441764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:45:54.441764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:45:54.441764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:45:54.441764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:45:54.441764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:45:54.441764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:54.441764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 254/518] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:54.442342 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac277d8-922f-42c8-9ad4-435d89301d66 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:54.489547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0920a99-4d88-4339-863c-8580585bb5dd None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:54.522888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd8e8e-bc45-40e0-a99e-636dae303cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:54.524118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd8e8e-bc45-40e0-a99e-636dae303cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:54.524460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd8e8e-bc45-40e0-a99e-636dae303cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:54.524746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd8e8e-bc45-40e0-a99e-636dae303cd1 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:54.525578 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd8e8e-bc45-40e0-a99e-636dae303cd1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:54.525886 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd8e8e-bc45-40e0-a99e-636dae303cd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:54.527344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd8e8e-bc45-40e0-a99e-636dae303cd1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:54.540989 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac277d8-922f-42c8-9ad4-435d89301d66 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:54.544682 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 265/519] 10.222.0.70 () {56 vars in 1139 bytes} [Thu Jun 16 18:45:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:54.545477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd8e8e-bc45-40e0-a99e-636dae303cd1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:54.558224 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:54.565273 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:54.576508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f513f1-a66c-4863-984a-659c8b7bb066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:54.577163 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f513f1-a66c-4863-984a-659c8b7bb066 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:54.577443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f513f1-a66c-4863-984a-659c8b7bb066 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:54.577744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f513f1-a66c-4863-984a-659c8b7bb066 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:54.578459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f513f1-a66c-4863-984a-659c8b7bb066 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:54.578787 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f513f1-a66c-4863-984a-659c8b7bb066 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:54.580143 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f513f1-a66c-4863-984a-659c8b7bb066 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:54.592078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 255/520] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:54.592859 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f513f1-a66c-4863-984a-659c8b7bb066 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:54.648908 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd8e8e-bc45-40e0-a99e-636dae303cd1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:54.683549 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2734c-27d9-45b1-b0a8-280b7c1d1a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:54.684334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2734c-27d9-45b1-b0a8-280b7c1d1a46 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:54.684615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2734c-27d9-45b1-b0a8-280b7c1d1a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:54.684886 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2734c-27d9-45b1-b0a8-280b7c1d1a46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:54.691526 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2734c-27d9-45b1-b0a8-280b7c1d1a46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:54.691636 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f513f1-a66c-4863-984a-659c8b7bb066 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:54.692966 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2734c-27d9-45b1-b0a8-280b7c1d1a46 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:54.708091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-41b2734c-27d9-45b1-b0a8-280b7c1d1a46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:45:54.708091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:54.708091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:54.708091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:54.708091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:54.708091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:54.708091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:54.708091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:54.708091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:54.708091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:54.708091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:54.708091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:54.708091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:45:54.708091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:45:54.708091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:45:54.708933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:45:54.708933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:54.708933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:54.708933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:54.708933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:54.708933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:54.708933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:54.708933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:54.708933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:54.708933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:54.708933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:54.708933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:54.708933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:54.708933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:54.708933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:54.708933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:54.708933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:54.709777 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:45:54.709777 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:45:54.709777 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:45:54.709777 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:45:54.709777 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:45:54.709777 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:45:54.709777 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:45:54.709777 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:54.709777 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 266/521] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:45:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:54.710384 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2734c-27d9-45b1-b0a8-280b7c1d1a46 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:54.725168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0c31f-0d38-4876-92d5-6658dc30d696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:54.725826 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0c31f-0d38-4876-92d5-6658dc30d696 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:54.726100 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0c31f-0d38-4876-92d5-6658dc30d696 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:54.726402 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0c31f-0d38-4876-92d5-6658dc30d696 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:54.727245 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0c31f-0d38-4876-92d5-6658dc30d696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:54.727547 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0c31f-0d38-4876-92d5-6658dc30d696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:54.728915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0c31f-0d38-4876-92d5-6658dc30d696 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:54.739950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 256/522] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:54.740733 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0c31f-0d38-4876-92d5-6658dc30d696 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:54.783695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:54.789644 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:54.815887 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2734c-27d9-45b1-b0a8-280b7c1d1a46 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:54.844006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0c31f-0d38-4876-92d5-6658dc30d696 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:54.848781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f9f20-947b-4059-a1d7-1906637606c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:54.849480 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f9f20-947b-4059-a1d7-1906637606c2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:54.849790 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f9f20-947b-4059-a1d7-1906637606c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:54.850096 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f9f20-947b-4059-a1d7-1906637606c2 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:54.851027 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f9f20-947b-4059-a1d7-1906637606c2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:54.851358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f9f20-947b-4059-a1d7-1906637606c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:54.852916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f9f20-947b-4059-a1d7-1906637606c2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:54.871011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 267/523] 10.222.0.70 () {56 vars in 1137 bytes} [Thu Jun 16 18:45:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:54.871796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f9f20-947b-4059-a1d7-1906637606c2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:54.878178 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689f6b7-27e2-44b4-ab23-0a578b8947bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:54.878851 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689f6b7-27e2-44b4-ab23-0a578b8947bc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:54.879132 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689f6b7-27e2-44b4-ab23-0a578b8947bc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:54.879402 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689f6b7-27e2-44b4-ab23-0a578b8947bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:54.950895 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689f6b7-27e2-44b4-ab23-0a578b8947bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:54.952368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689f6b7-27e2-44b4-ab23-0a578b8947bc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:54.974792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f9f20-947b-4059-a1d7-1906637606c2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:54.997627 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:55.001455 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:55.010107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47522a6-c8c9-4d55-a915-4ad5e3e5775e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:55.010801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47522a6-c8c9-4d55-a915-4ad5e3e5775e None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:55.011083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47522a6-c8c9-4d55-a915-4ad5e3e5775e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:55.011359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47522a6-c8c9-4d55-a915-4ad5e3e5775e None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:55.012053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47522a6-c8c9-4d55-a915-4ad5e3e5775e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:55.013487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47522a6-c8c9-4d55-a915-4ad5e3e5775e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:55.032623 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 268/524] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:45:54 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:45:55.033425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47522a6-c8c9-4d55-a915-4ad5e3e5775e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:55.072571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 257/525] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:55.073900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689f6b7-27e2-44b4-ab23-0a578b8947bc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:55.150961 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47522a6-c8c9-4d55-a915-4ad5e3e5775e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:55.189338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70713f-8672-410f-a1b5-6b4b0786fa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:55.190006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70713f-8672-410f-a1b5-6b4b0786fa00 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:55.190283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70713f-8672-410f-a1b5-6b4b0786fa00 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:55.190446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689f6b7-27e2-44b4-ab23-0a578b8947bc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:55.190592 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70713f-8672-410f-a1b5-6b4b0786fa00 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:55.191356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba70713f-8672-410f-a1b5-6b4b0786fa00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:55.191664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba70713f-8672-410f-a1b5-6b4b0786fa00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:55.193096 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba70713f-8672-410f-a1b5-6b4b0786fa00 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:55.218459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 269/526] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:45:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:55.219281 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70713f-8672-410f-a1b5-6b4b0786fa00 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:55.234263 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdf53d-1688-46e0-bb7b-e6419c6b8484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:55.235100 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdf53d-1688-46e0-bb7b-e6419c6b8484 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:55.235383 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdf53d-1688-46e0-bb7b-e6419c6b8484 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:55.235659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdf53d-1688-46e0-bb7b-e6419c6b8484 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:55.246251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cdf53d-1688-46e0-bb7b-e6419c6b8484 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:55.247881 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cdf53d-1688-46e0-bb7b-e6419c6b8484 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:55.263614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e9cdf53d-1688-46e0-bb7b-e6419c6b8484 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:45:55.263614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:55.263614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:55.263614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:55.263614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:55.263614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:55.263614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:55.263614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:55.263614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:55.263614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:55.263614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:55.263614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:55.263614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:45:55.263614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:45:55.263614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:45:55.264912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:45:55.264912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:55.264912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:55.264912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:55.264912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:55.264912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:55.264912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:55.264912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:55.264912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:55.264912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:55.264912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:55.264912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:55.264912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:55.264912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:55.264912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:55.264912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:55.264912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:55.265746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:45:55.265746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:45:55.265746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:45:55.265746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:45:55.265746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:45:55.265746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:45:55.265746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:45:55.265746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:55.265746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 258/527] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:45:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:55.265746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdf53d-1688-46e0-bb7b-e6419c6b8484 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:55.270995 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:55.278721 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:55.339785 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70713f-8672-410f-a1b5-6b4b0786fa00 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:55.371548 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdf53d-1688-46e0-bb7b-e6419c6b8484 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:55.379232 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275ac1e-b9ec-40a0-9d58-68369f5f1577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:55.379891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275ac1e-b9ec-40a0-9d58-68369f5f1577 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:55.380295 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275ac1e-b9ec-40a0-9d58-68369f5f1577 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:55.380621 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275ac1e-b9ec-40a0-9d58-68369f5f1577 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:55.409483 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61307f80-d806-4c1f-99bb-26b4eab0112b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:55.410291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61307f80-d806-4c1f-99bb-26b4eab0112b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:55.410599 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61307f80-d806-4c1f-99bb-26b4eab0112b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:55.410876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61307f80-d806-4c1f-99bb-26b4eab0112b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:55.422029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61307f80-d806-4c1f-99bb-26b4eab0112b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:55.423514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61307f80-d806-4c1f-99bb-26b4eab0112b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:55.440743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-61307f80-d806-4c1f-99bb-26b4eab0112b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:55.440743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:55.440743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:55.440743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:55.440743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:55.440743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:55.440743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:55.440743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:55.440743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:55.440743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:55.440743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:55.440743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:55.440743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:45:55.440743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:45:55.440743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:45:55.441855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:45:55.441855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:55.441855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:55.441855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:55.441855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:55.441855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:55.441855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:55.441855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:55.441855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:55.441855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:55.441855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:55.441855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:55.441855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:55.441855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:55.441855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:55.441855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:55.441855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:55.442752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:45:55.442752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:45:55.442752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:55.442752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:55.442752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 259/528] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:55.443028 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61307f80-d806-4c1f-99bb-26b4eab0112b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:55.463492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275ac1e-b9ec-40a0-9d58-68369f5f1577 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:55.465314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275ac1e-b9ec-40a0-9d58-68369f5f1577 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:55.506713 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:55.512389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:55.559429 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61307f80-d806-4c1f-99bb-26b4eab0112b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:55.576298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 270/529] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:55.576298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275ac1e-b9ec-40a0-9d58-68369f5f1577 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:55.599535 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb4847-4670-46a8-b382-4873fccd18f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:55.600305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb4847-4670-46a8-b382-4873fccd18f1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:55.600583 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb4847-4670-46a8-b382-4873fccd18f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:55.600853 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb4847-4670-46a8-b382-4873fccd18f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:55.601793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb4847-4670-46a8-b382-4873fccd18f1 None admin] RBAC: Adding query filter params (role.id=c123034c11cc4d9ea6cab00e05bdcd53, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=8342da838fe4462b9de2817ad3ac4ff7) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:55.602275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb4847-4670-46a8-b382-4873fccd18f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:55.603915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb4847-4670-46a8-b382-4873fccd18f1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:55.616425 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 260/530] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:55 2022] GET /identity/v3/role_assignments?user.id=8342da838fe4462b9de2817ad3ac4ff7&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=c123034c11cc4d9ea6cab00e05bdcd53 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:55.616425 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb4847-4670-46a8-b382-4873fccd18f1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:55.692635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275ac1e-b9ec-40a0-9d58-68369f5f1577 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:55.733850 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36fa17-c830-4d34-b44b-36185f15861b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:55.735051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36fa17-c830-4d34-b44b-36185f15861b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:55.735554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36fa17-c830-4d34-b44b-36185f15861b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:55.736018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36fa17-c830-4d34-b44b-36185f15861b None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:55.737030 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36fa17-c830-4d34-b44b-36185f15861b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:55.737544 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36fa17-c830-4d34-b44b-36185f15861b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:55.738167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb4847-4670-46a8-b382-4873fccd18f1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:55.739430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36fa17-c830-4d34-b44b-36185f15861b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:55.754561 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 271/531] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:45:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:55.755992 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36fa17-c830-4d34-b44b-36185f15861b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:55.761067 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:55.763735 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:55.781852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7ec5a-680c-4161-967d-9e84b7568578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:55.782889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7ec5a-680c-4161-967d-9e84b7568578 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:55.783171 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7ec5a-680c-4161-967d-9e84b7568578 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:55.783840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7ec5a-680c-4161-967d-9e84b7568578 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:55.784623 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7ec5a-680c-4161-967d-9e84b7568578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:55.784988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7ec5a-680c-4161-967d-9e84b7568578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:55.786415 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7ec5a-680c-4161-967d-9e84b7568578 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:55.800481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 261/532] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:55.801394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7ec5a-680c-4161-967d-9e84b7568578 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:55.888251 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36fa17-c830-4d34-b44b-36185f15861b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:55.931859 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7ec5a-680c-4161-967d-9e84b7568578 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:55.933022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c196fd8-404c-48e6-bc87-46010caf0335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:55.934096 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c196fd8-404c-48e6-bc87-46010caf0335 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:55.935340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c196fd8-404c-48e6-bc87-46010caf0335 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:55.935626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c196fd8-404c-48e6-bc87-46010caf0335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:55.950342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c196fd8-404c-48e6-bc87-46010caf0335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:55.952415 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c196fd8-404c-48e6-bc87-46010caf0335 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:55.972152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd947fe-80f2-4af1-94b1-1567e0b9e52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:55.972893 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd947fe-80f2-4af1-94b1-1567e0b9e52a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:55.973293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd947fe-80f2-4af1-94b1-1567e0b9e52a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:55.973629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd947fe-80f2-4af1-94b1-1567e0b9e52a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:55.974604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd947fe-80f2-4af1-94b1-1567e0b9e52a None admin] RBAC: Adding query filter params (role.id=c123034c11cc4d9ea6cab00e05bdcd53, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=8563a25d16134d639a2a8849976fed9a) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:55.974998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd947fe-80f2-4af1-94b1-1567e0b9e52a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:55.975832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-3c196fd8-404c-48e6-bc87-46010caf0335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:55.975832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:55.975832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:55.975832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:55.975832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:55.975832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:55.975832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:55.975832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:55.975832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:55.975832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:55.975832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:55.975832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:55.975832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:45:55.975832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:45:55.975832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:45:55.977410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:45:55.977410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:55.977410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:55.977410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:55.977410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:55.977410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:55.977410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:55.977410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:55.977410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:55.977410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:55.977410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:55.977410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:55.977410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:55.977410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:55.977410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:55.977410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:55.977410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:55.978717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:45:55.978717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:45:55.978717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:55.978717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:55.979142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd947fe-80f2-4af1-94b1-1567e0b9e52a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:55.980769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 272/533] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:55.982173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c196fd8-404c-48e6-bc87-46010caf0335 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:55.987613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 262/534] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:55 2022] GET /identity/v3/role_assignments?user.id=8563a25d16134d639a2a8849976fed9a&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=c123034c11cc4d9ea6cab00e05bdcd53 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:55.988592 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd947fe-80f2-4af1-94b1-1567e0b9e52a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:56.076246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:56.098966 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:56.186127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c196fd8-404c-48e6-bc87-46010caf0335 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:56.190149 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd947fe-80f2-4af1-94b1-1567e0b9e52a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:56.264563 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84035a-4aa8-483c-81dc-628f16aef047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:56.265751 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84035a-4aa8-483c-81dc-628f16aef047 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:56.266174 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84035a-4aa8-483c-81dc-628f16aef047 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:56.266612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84035a-4aa8-483c-81dc-628f16aef047 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:56.268537 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb07447-39b1-4e40-99cd-f91de0602afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:56.269456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb07447-39b1-4e40-99cd-f91de0602afd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:56.269879 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb07447-39b1-4e40-99cd-f91de0602afd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:56.270295 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb07447-39b1-4e40-99cd-f91de0602afd None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:56.271487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb07447-39b1-4e40-99cd-f91de0602afd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:56.271952 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb07447-39b1-4e40-99cd-f91de0602afd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:56.274641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb07447-39b1-4e40-99cd-f91de0602afd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:56.290793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 263/535] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:56.292105 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84035a-4aa8-483c-81dc-628f16aef047 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:56.293601 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84035a-4aa8-483c-81dc-628f16aef047 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:56.302188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb07447-39b1-4e40-99cd-f91de0602afd None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:56.315207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-5a84035a-4aa8-483c-81dc-628f16aef047 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:45:56.315207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:56.315207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:56.315207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:56.315207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:56.315207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:56.315207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:56.315207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:56.315207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:56.315207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:56.315207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:56.315207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:56.315207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:45:56.315207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:45:56.315207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:45:56.316118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:45:56.316118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:56.316118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:56.316118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:56.316118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:56.316118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:56.316118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:56.316118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:56.316118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:56.316118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:56.316118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:56.316118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:56.316118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:56.316118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:56.316118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:56.316118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:56.316118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:56.316965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:45:56.316965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:45:56.316965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:45:56.316965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:45:56.316965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:45:56.316965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:45:56.316965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:45:56.316965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:56.316965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 273/536] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:45:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:56.322891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84035a-4aa8-483c-81dc-628f16aef047 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:56.431595 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84035a-4aa8-483c-81dc-628f16aef047 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:56.449375 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:56.470424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb67103-f443-4473-a8aa-298312e24158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:56.471717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb67103-f443-4473-a8aa-298312e24158 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:56.472200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb67103-f443-4473-a8aa-298312e24158 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:56.472706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb67103-f443-4473-a8aa-298312e24158 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:56.474321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb67103-f443-4473-a8aa-298312e24158 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:56.474880 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb67103-f443-4473-a8aa-298312e24158 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:56.477632 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb67103-f443-4473-a8aa-298312e24158 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:56.490634 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 274/537] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:45:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:45:56.497108 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb67103-f443-4473-a8aa-298312e24158 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:56.550319 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb07447-39b1-4e40-99cd-f91de0602afd None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:56.561484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:56.625692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203c102-877b-43ae-986a-49500d384864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:56.626499 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203c102-877b-43ae-986a-49500d384864 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:56.626811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203c102-877b-43ae-986a-49500d384864 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:56.627089 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203c102-877b-43ae-986a-49500d384864 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:56.644057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203c102-877b-43ae-986a-49500d384864 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:56.647394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb67103-f443-4473-a8aa-298312e24158 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:56.648820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203c102-877b-43ae-986a-49500d384864 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:56.670147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-1203c102-877b-43ae-986a-49500d384864 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:45:56.670147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:56.670147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:56.670147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:56.670147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:56.670147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:56.670147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:56.670147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:56.670147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:56.670147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:56.670147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:56.670147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:56.670147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:45:56.670147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:45:56.670147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:45:56.671056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:45:56.671056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:56.671056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:56.671056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:56.671056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:56.671056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:56.671056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:56.671056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:56.671056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:56.671056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:56.671056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:56.671056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:56.671056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:56.671056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:56.671056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:56.671056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:56.671056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:56.672527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:45:56.672527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:45:56.672527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:45:56.672527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:45:56.672527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:45:56.672527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:45:56.672527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:45:56.672527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:56.672527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 264/538] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:45:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:56.683103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203c102-877b-43ae-986a-49500d384864 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:56.717738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca7c0a-38e1-4c8e-97ab-8b5272856b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:56.719976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca7c0a-38e1-4c8e-97ab-8b5272856b04 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:56.720254 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca7c0a-38e1-4c8e-97ab-8b5272856b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:56.720527 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca7c0a-38e1-4c8e-97ab-8b5272856b04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:56.721361 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca7c0a-38e1-4c8e-97ab-8b5272856b04 None admin] RBAC: Adding query filter params (role.id=c123034c11cc4d9ea6cab00e05bdcd53, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=02a83f6e7aa448d6984a78914c1748cf) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:56.721670 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca7c0a-38e1-4c8e-97ab-8b5272856b04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:56.728404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca7c0a-38e1-4c8e-97ab-8b5272856b04 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:56.743587 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 275/539] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:56 2022] GET /identity/v3/role_assignments?user.id=02a83f6e7aa448d6984a78914c1748cf&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=c123034c11cc4d9ea6cab00e05bdcd53 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:56.843138 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203c102-877b-43ae-986a-49500d384864 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:56.890627 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21a2e3-3dce-45b2-a934-6c8fb85a5534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:56.891309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21a2e3-3dce-45b2-a934-6c8fb85a5534 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:56.891596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21a2e3-3dce-45b2-a934-6c8fb85a5534 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:56.891875 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21a2e3-3dce-45b2-a934-6c8fb85a5534 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:56.892675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21a2e3-3dce-45b2-a934-6c8fb85a5534 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:56.893240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21a2e3-3dce-45b2-a934-6c8fb85a5534 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:56.894716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21a2e3-3dce-45b2-a934-6c8fb85a5534 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:56.927163 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 265/540] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:45:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:56.933368 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca7c0a-38e1-4c8e-97ab-8b5272856b04 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:57.016798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:57.094987 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca7c0a-38e1-4c8e-97ab-8b5272856b04 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:57.144916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177f569-2d30-4def-9136-530993774daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:57.145580 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177f569-2d30-4def-9136-530993774daa None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:57.145865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177f569-2d30-4def-9136-530993774daa None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:57.146146 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177f569-2d30-4def-9136-530993774daa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:57.147013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177f569-2d30-4def-9136-530993774daa None admin] RBAC: Adding query filter params (role.id=c123034c11cc4d9ea6cab00e05bdcd53, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=a47de73e3f3b4979bff83140bf287275) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:57.147331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177f569-2d30-4def-9136-530993774daa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:57.149343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177f569-2d30-4def-9136-530993774daa None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:57.159093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 276/541] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:45:56 2022] GET /identity/v3/role_assignments?user.id=a47de73e3f3b4979bff83140bf287275&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=c123034c11cc4d9ea6cab00e05bdcd53 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:57.506305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdd0b0-0410-423b-b10b-f55fa23bd477 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:57.507364 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdd0b0-0410-423b-b10b-f55fa23bd477 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:57.508393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdd0b0-0410-423b-b10b-f55fa23bd477 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:57.510336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 266/542] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:57.515397 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac0273-0731-48b4-8dbc-2e17c22a3ee2 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:57.515860 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac0273-0731-48b4-8dbc-2e17c22a3ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:57.516056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac0273-0731-48b4-8dbc-2e17c22a3ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:57.538622 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d8ac0273-0731-48b4-8dbc-2e17c22a3ee2 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:57.540561 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d8ac0273-0731-48b4-8dbc-2e17c22a3ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:57.645666 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 277/543] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:57.685339 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdd0b0-0410-423b-b10b-f55fa23bd477 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:57.721258 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:57.836806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdd0b0-0410-423b-b10b-f55fa23bd477 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:57.877260 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084862c1-1f52-49a4-b957-89577ab4cb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:57.878064 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084862c1-1f52-49a4-b957-89577ab4cb9a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:57.878344 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084862c1-1f52-49a4-b957-89577ab4cb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:57.878641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084862c1-1f52-49a4-b957-89577ab4cb9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:57.891963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084862c1-1f52-49a4-b957-89577ab4cb9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:57.901236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-084862c1-1f52-49a4-b957-89577ab4cb9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:45:57.901236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:57.901236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:57.901236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:57.901236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:57.901236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:57.901236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:57.901236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:57.901236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:57.901236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:57.901236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:57.901236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:57.901236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:57.901236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:57.901236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:57.901236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:57.902168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:57.902168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:57.902168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:57.902168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:57.902168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:57.902168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:57.902168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:57.902168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:57.902168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:57.902168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:57.902168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:57.902168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:57.902168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:57.902168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:57.902168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:57.902168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:57.903039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:57.903039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:57.903039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:57.903039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:57.903039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:57.903039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:57.903039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:57.903039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:57.903039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:57.903039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:57.903039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:45:57.903039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:57.903039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 267/544] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:45:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:57.910135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac0273-0731-48b4-8dbc-2e17c22a3ee2 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:57.985225 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:58.094851 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac0273-0731-48b4-8dbc-2e17c22a3ee2 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:58.151675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c3f6d-1f2f-47bc-a3f1-ca06e0f11aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:58.152357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c3f6d-1f2f-47bc-a3f1-ca06e0f11aad None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:58.152644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c3f6d-1f2f-47bc-a3f1-ca06e0f11aad None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:58.152925 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c3f6d-1f2f-47bc-a3f1-ca06e0f11aad None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:58.153679 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c3f6d-1f2f-47bc-a3f1-ca06e0f11aad None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:58.153990 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c3f6d-1f2f-47bc-a3f1-ca06e0f11aad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:58.155399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c3f6d-1f2f-47bc-a3f1-ca06e0f11aad None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:58.156133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70bf61-7478-405e-a446-b53950f1bc26 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:58.156429 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70bf61-7478-405e-a446-b53950f1bc26 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:58.156713 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70bf61-7478-405e-a446-b53950f1bc26 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:58.158361 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 268/545] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:58.163282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02675cfd-f87d-44a2-917b-65a2e1c61cc0 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:58.163570 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02675cfd-f87d-44a2-917b-65a2e1c61cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:58.164382 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02675cfd-f87d-44a2-917b-65a2e1c61cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:58.173970 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 278/546] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:45:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:58.179477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c3f6d-1f2f-47bc-a3f1-ca06e0f11aad None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:58.183852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-02675cfd-f87d-44a2-917b-65a2e1c61cc0 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:58.186019 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-02675cfd-f87d-44a2-917b-65a2e1c61cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:58.284744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 269/547] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:45:58.334425 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02675cfd-f87d-44a2-917b-65a2e1c61cc0 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:58.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:58.344136 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c3f6d-1f2f-47bc-a3f1-ca06e0f11aad None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:58.365778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:58.417610 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b310ce-5896-421e-b4d5-29e5154b531d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:58.418421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b310ce-5896-421e-b4d5-29e5154b531d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:58.418735 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b310ce-5896-421e-b4d5-29e5154b531d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:58.419012 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b310ce-5896-421e-b4d5-29e5154b531d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:58.440808 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b310ce-5896-421e-b4d5-29e5154b531d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:58.442279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b310ce-5896-421e-b4d5-29e5154b531d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:58.470183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-58b310ce-5896-421e-b4d5-29e5154b531d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:58.470183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:58.470183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:58.470183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:58.470183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:58.470183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:58.470183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:58.470183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:58.470183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:58.470183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:58.470183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:58.470183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:58.470183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:45:58.470183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:45:58.470183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:45:58.471153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:45:58.471153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:58.471153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:58.471153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:58.471153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:58.471153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:58.471153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:58.471153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:58.471153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:58.471153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:58.471153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:58.471153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:58.471153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:58.471153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:58.471153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:58.471153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:58.471153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:58.472001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:45:58.472001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:45:58.472001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:58.472001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:58.472001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 279/548] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:58.478938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b310ce-5896-421e-b4d5-29e5154b531d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:58.541337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02675cfd-f87d-44a2-917b-65a2e1c61cc0 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:58.581074 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63f8f0-c0de-4419-b44d-7aab8aadb2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:58.581878 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63f8f0-c0de-4419-b44d-7aab8aadb2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:58.582141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63f8f0-c0de-4419-b44d-7aab8aadb2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:58.582415 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63f8f0-c0de-4419-b44d-7aab8aadb2d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:58.595508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63f8f0-c0de-4419-b44d-7aab8aadb2d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:58.605393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ca63f8f0-c0de-4419-b44d-7aab8aadb2d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:45:58.605393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:58.605393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:58.605393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:58.605393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:58.605393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:58.605393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:58.605393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:58.605393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:58.605393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:58.605393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:58.605393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:58.605393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:58.605393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:58.605393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:58.605393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:58.606514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:58.606514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:58.606514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:58.606514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:58.606514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:58.606514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:58.606514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:58.606514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:58.606514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:58.606514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:58.606514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:58.606514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:58.606514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:58.606514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:58.606514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:58.606514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:58.607935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:58.607935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:58.607935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:58.607935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:58.607935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:58.607935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:58.607935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:58.607935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:58.607935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:58.607935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:58.607935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:45:58.607935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:58.607935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 270/549] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:45:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:45:58.608789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44750620-f5e2-498c-b325-c528d9104a91 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:58.608789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44750620-f5e2-498c-b325-c528d9104a91 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:58.608789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44750620-f5e2-498c-b325-c528d9104a91 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:58.614528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 271/550] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:58.615309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44750620-f5e2-498c-b325-c528d9104a91 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:58.630081 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b310ce-5896-421e-b4d5-29e5154b531d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:58.677266 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f384d0-b1a2-4f1c-bd19-097c136c2507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:58.677943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f384d0-b1a2-4f1c-bd19-097c136c2507 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:58.678268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f384d0-b1a2-4f1c-bd19-097c136c2507 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:58.678543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f384d0-b1a2-4f1c-bd19-097c136c2507 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:58.679897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f384d0-b1a2-4f1c-bd19-097c136c2507 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:58.680222 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f384d0-b1a2-4f1c-bd19-097c136c2507 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:58.682325 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f384d0-b1a2-4f1c-bd19-097c136c2507 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:58.695565 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 280/551] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:58.697045 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c56347-2034-41d7-a829-99044442a6b3 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:58.697331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c56347-2034-41d7-a829-99044442a6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:58.697606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c56347-2034-41d7-a829-99044442a6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:58.713976 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:58.714465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-35c56347-2034-41d7-a829-99044442a6b3 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:58.715742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-35c56347-2034-41d7-a829-99044442a6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:58.717903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:58.750741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44750620-f5e2-498c-b325-c528d9104a91 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:58.790659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 281/552] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:58.791876 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c56347-2034-41d7-a829-99044442a6b3 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:58.798041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93a5e9-e7a4-4769-a45a-ad388beeeef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:58.799212 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93a5e9-e7a4-4769-a45a-ad388beeeef5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:58.799699 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93a5e9-e7a4-4769-a45a-ad388beeeef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:58.800177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93a5e9-e7a4-4769-a45a-ad388beeeef5 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:58.801130 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b93a5e9-e7a4-4769-a45a-ad388beeeef5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:58.801749 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b93a5e9-e7a4-4769-a45a-ad388beeeef5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:58.805073 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b93a5e9-e7a4-4769-a45a-ad388beeeef5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:58.823271 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 272/553] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:45:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:58.824034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93a5e9-e7a4-4769-a45a-ad388beeeef5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:58.940786 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93a5e9-e7a4-4769-a45a-ad388beeeef5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:58.941929 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c56347-2034-41d7-a829-99044442a6b3 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:58.986193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8baca-7060-4bbc-a5c1-d4ffcaff2985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:58.987192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8baca-7060-4bbc-a5c1-d4ffcaff2985 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:58.987476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8baca-7060-4bbc-a5c1-d4ffcaff2985 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:58.987747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8baca-7060-4bbc-a5c1-d4ffcaff2985 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:58.990250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3628c-6a60-4b19-b751-602621bc7fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:58.991086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:58.993081 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3628c-6a60-4b19-b751-602621bc7fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:58.994735 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3628c-6a60-4b19-b751-602621bc7fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:58.995401 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3628c-6a60-4b19-b751-602621bc7fe2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:59.000626 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8baca-7060-4bbc-a5c1-d4ffcaff2985 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:45:59.006218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3628c-6a60-4b19-b751-602621bc7fe2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:59.008875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3628c-6a60-4b19-b751-602621bc7fe2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:59.009335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f2e8baca-7060-4bbc-a5c1-d4ffcaff2985 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:45:59.009335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:59.009335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:59.009335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:59.009335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:59.009335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:59.009335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:59.009335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:59.009335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:59.009335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:59.009335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:59.009335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:59.009335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:45:59.009335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:45:59.009335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:45:59.009335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:45:59.010360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:45:59.010360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:45:59.010360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:45:59.010360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:45:59.010360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:59.010360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:59.010360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:59.010360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:59.010360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:59.010360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:59.010360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:59.010360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:59.010360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:59.010360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:59.010360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:59.010360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:59.011264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:59.011264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:59.011264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:59.011264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:59.011264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:45:59.011264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:45:59.011264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:45:59.011264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:45:59.011264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:45:59.011264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:45:59.011264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:45:59.011264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:59.014724 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 273/554] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:45:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:59.016268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8baca-7060-4bbc-a5c1-d4ffcaff2985 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:59.018494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:59.038218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ddd3628c-6a60-4b19-b751-602621bc7fe2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:45:59.038218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:59.038218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:59.038218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:59.038218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:59.038218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:59.038218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:59.038218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:59.038218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:59.038218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:59.038218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:59.038218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:59.038218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:45:59.038218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:45:59.038218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:45:59.038218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:59.040092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:59.041154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:59.041154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:59.041154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:45:59.041154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:45:59.041154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:45:59.041154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:45:59.041154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:45:59.041154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:45:59.041154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:45:59.041154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:59.043902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 282/555] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:45:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:59.045351 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3628c-6a60-4b19-b751-602621bc7fe2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:59.141240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8baca-7060-4bbc-a5c1-d4ffcaff2985 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:59.156156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3628c-6a60-4b19-b751-602621bc7fe2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:59.179919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81575b6f-8305-4fab-96b9-645646e0a5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:59.181095 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81575b6f-8305-4fab-96b9-645646e0a5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:59.181554 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81575b6f-8305-4fab-96b9-645646e0a5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:59.182060 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81575b6f-8305-4fab-96b9-645646e0a5c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:59.192722 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab42d64-a28d-4357-9e78-33f6357178ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:59.193422 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab42d64-a28d-4357-9e78-33f6357178ea None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:59.193785 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab42d64-a28d-4357-9e78-33f6357178ea None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:59.194179 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab42d64-a28d-4357-9e78-33f6357178ea None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:59.194972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab42d64-a28d-4357-9e78-33f6357178ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:59.195407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81575b6f-8305-4fab-96b9-645646e0a5c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:59.195514 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab42d64-a28d-4357-9e78-33f6357178ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:59.197216 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab42d64-a28d-4357-9e78-33f6357178ea None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:59.197529 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81575b6f-8305-4fab-96b9-645646e0a5c3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:59.212004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 283/556] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:45:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:59.213084 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab42d64-a28d-4357-9e78-33f6357178ea None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:59.219141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-81575b6f-8305-4fab-96b9-645646e0a5c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:59.219141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:59.219141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:59.219141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:59.219141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:59.219141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:59.219141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:59.219141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:59.219141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:59.219141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:59.219141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:59.219141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:59.219141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:45:59.219141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:45:59.219141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:45:59.220058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:45:59.220058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:59.220058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:59.220058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:59.220058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:59.220058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:59.220058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:59.220058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:59.220058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:59.220058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:59.220058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:59.220058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:59.220058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:59.220058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:59.220058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:59.220058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:59.220058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:59.221465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:45:59.221465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:45:59.221465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:59.221465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:59.223800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 274/557] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:59.224949 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81575b6f-8305-4fab-96b9-645646e0a5c3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:59.277410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:59.289354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:59.322620 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab42d64-a28d-4357-9e78-33f6357178ea None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:59.353498 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81575b6f-8305-4fab-96b9-645646e0a5c3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:59.360432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ec5c4-cff8-4fba-abba-5bc17c3b73d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:59.361187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ec5c4-cff8-4fba-abba-5bc17c3b73d1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:59.361531 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ec5c4-cff8-4fba-abba-5bc17c3b73d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:59.361873 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ec5c4-cff8-4fba-abba-5bc17c3b73d1 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:59.362795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ec5c4-cff8-4fba-abba-5bc17c3b73d1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:59.363165 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ec5c4-cff8-4fba-abba-5bc17c3b73d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:59.364671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ec5c4-cff8-4fba-abba-5bc17c3b73d1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:59.383591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 284/558] 10.222.0.70 () {56 vars in 1135 bytes} [Thu Jun 16 18:45:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:59.384645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ec5c4-cff8-4fba-abba-5bc17c3b73d1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:59.392226 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfafce0-c6fa-4f39-9143-83a2d46d93c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:59.393295 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfafce0-c6fa-4f39-9143-83a2d46d93c6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:59.393615 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfafce0-c6fa-4f39-9143-83a2d46d93c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:59.393903 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfafce0-c6fa-4f39-9143-83a2d46d93c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:59.406825 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfafce0-c6fa-4f39-9143-83a2d46d93c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:59.408280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfafce0-c6fa-4f39-9143-83a2d46d93c6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:59.427320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-9bfafce0-c6fa-4f39-9143-83a2d46d93c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:59.427320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:59.427320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:59.427320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:59.427320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:59.427320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:59.427320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:59.427320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:59.427320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:59.427320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:59.427320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:59.427320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:59.427320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:45:59.427320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:45:59.427320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:45:59.428327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:45:59.428327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:59.428327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:59.428327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:59.428327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:59.428327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:59.428327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:59.428327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:59.428327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:59.428327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:59.428327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:59.428327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:59.428327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:59.428327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:59.428327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:59.428327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:59.428327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:59.429280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:45:59.429280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:45:59.429280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:45:59.429280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:45:59.429280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 275/559] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:59.429580 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfafce0-c6fa-4f39-9143-83a2d46d93c6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:59.504667 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ec5c4-cff8-4fba-abba-5bc17c3b73d1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:59.524705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:59.536862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:59.540458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfafce0-c6fa-4f39-9143-83a2d46d93c6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:59.543792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025db467-11fb-4d58-abe6-c3e352284c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:59.544556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025db467-11fb-4d58-abe6-c3e352284c28 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:59.544908 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025db467-11fb-4d58-abe6-c3e352284c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:59.545256 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025db467-11fb-4d58-abe6-c3e352284c28 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:59.546060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025db467-11fb-4d58-abe6-c3e352284c28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:59.546439 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025db467-11fb-4d58-abe6-c3e352284c28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:59.547960 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025db467-11fb-4d58-abe6-c3e352284c28 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:59.563496 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 285/560] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:59.565000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630cd29-e1ab-4f08-a9c4-dd9c6a23c256 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:59.565566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630cd29-e1ab-4f08-a9c4-dd9c6a23c256 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:59.565843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630cd29-e1ab-4f08-a9c4-dd9c6a23c256 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:59.567270 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 286/561] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:45:59.567800 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630cd29-e1ab-4f08-a9c4-dd9c6a23c256 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:59.588372 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194adb3-ae07-4dfb-83a3-8c77b270afda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:59.589341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194adb3-ae07-4dfb-83a3-8c77b270afda None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:59.590049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194adb3-ae07-4dfb-83a3-8c77b270afda None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:59.590408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194adb3-ae07-4dfb-83a3-8c77b270afda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:59.669309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5194adb3-ae07-4dfb-83a3-8c77b270afda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:59.670899 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5194adb3-ae07-4dfb-83a3-8c77b270afda None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:59.680618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630cd29-e1ab-4f08-a9c4-dd9c6a23c256 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:59.717429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3a735-8bc6-45c1-af9a-f4c802cd0667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:59.718106 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3a735-8bc6-45c1-af9a-f4c802cd0667 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:59.718389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3a735-8bc6-45c1-af9a-f4c802cd0667 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:59.718702 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3a735-8bc6-45c1-af9a-f4c802cd0667 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:59.719420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3a735-8bc6-45c1-af9a-f4c802cd0667 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:45:59.720038 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3a735-8bc6-45c1-af9a-f4c802cd0667 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:59.721096 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3a735-8bc6-45c1-af9a-f4c802cd0667 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:59.738587 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 287/562] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:45:59.738587 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3a735-8bc6-45c1-af9a-f4c802cd0667 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:59.780946 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:59.784658 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:59.794375 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 276/563] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:45:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:45:59.794375 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15ac0b-ecf6-497c-8552-a9217463a60d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:59.794375 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15ac0b-ecf6-497c-8552-a9217463a60d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:59.794375 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15ac0b-ecf6-497c-8552-a9217463a60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:59.809863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4b15ac0b-ecf6-497c-8552-a9217463a60d None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:45:59.811128 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4b15ac0b-ecf6-497c-8552-a9217463a60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:59.860280 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3a735-8bc6-45c1-af9a-f4c802cd0667 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:59.879431 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 277/564] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:45:59.880457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15ac0b-ecf6-497c-8552-a9217463a60d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:59.897386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1106c33-b583-4cdc-a319-517687f43047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:59.899287 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1106c33-b583-4cdc-a319-517687f43047 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:59.899949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1106c33-b583-4cdc-a319-517687f43047 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:59.900324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1106c33-b583-4cdc-a319-517687f43047 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:59.908304 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1106c33-b583-4cdc-a319-517687f43047 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:59.909806 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1106c33-b583-4cdc-a319-517687f43047 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:59.924879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-f1106c33-b583-4cdc-a319-517687f43047 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:45:59.924879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:45:59.924879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:45:59.924879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:45:59.924879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:45:59.924879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:45:59.924879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:45:59.924879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:45:59.924879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:45:59.924879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:45:59.924879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:45:59.924879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:45:59.924879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:45:59.924879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:45:59.924879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:45:59.924879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:45:59.925982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:45:59.927135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:45:59.927135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:45:59.927135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:45:59.927135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:45:59.927135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:45:59.927135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:45:59.927135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:45:59.927135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:45:59.927135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:45:59.927135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:45:59.927135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 288/565] 10.222.0.70 () {56 vars in 1093 bytes} [Thu Jun 16 18:45:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:45:59.927855 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1106c33-b583-4cdc-a319-517687f43047 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:59.985912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15ac0b-ecf6-497c-8552-a9217463a60d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:00.015088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:00.022145 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6debd-e0d9-47e4-bf91-f33da5bd6613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:00.023080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6debd-e0d9-47e4-bf91-f33da5bd6613 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:00.023435 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6debd-e0d9-47e4-bf91-f33da5bd6613 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:00.023717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6debd-e0d9-47e4-bf91-f33da5bd6613 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:00.028275 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:00.031309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6debd-e0d9-47e4-bf91-f33da5bd6613 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:00.032349 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1106c33-b583-4cdc-a319-517687f43047 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:00.032697 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6debd-e0d9-47e4-bf91-f33da5bd6613 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:00.047397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-6af6debd-e0d9-47e4-bf91-f33da5bd6613 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:46:00.047397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:00.047397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:00.047397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:00.047397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:00.047397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:00.047397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:00.047397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:00.047397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:00.047397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:00.047397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:00.047397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:00.047397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:00.047397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:00.047397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:00.047397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:00.048490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:00.049467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:00.049467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:00.049467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:00.049467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:00.049467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:00.049467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:00.049467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:00.049467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:00.049467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:46:00.049467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:00.049467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 278/566] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:45:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:00.050064 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e7d51-bd83-4bd0-9a69-0b6475c55585 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:00.050064 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e7d51-bd83-4bd0-9a69-0b6475c55585 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:00.050532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e7d51-bd83-4bd0-9a69-0b6475c55585 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:00.051722 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 279/567] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:00.052292 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e7d51-bd83-4bd0-9a69-0b6475c55585 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:00.070981 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d353bbb-4cd5-426d-b802-ef647d4d667f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:00.071774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d353bbb-4cd5-426d-b802-ef647d4d667f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:00.072058 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d353bbb-4cd5-426d-b802-ef647d4d667f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:00.072332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d353bbb-4cd5-426d-b802-ef647d4d667f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:00.082183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d353bbb-4cd5-426d-b802-ef647d4d667f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:00.083656 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d353bbb-4cd5-426d-b802-ef647d4d667f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:00.103053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-7d353bbb-4cd5-426d-b802-ef647d4d667f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:00.103053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:00.103053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:00.103053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:00.103053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:00.103053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:00.103053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:00.103053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:00.103053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:00.103053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:00.103053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:00.103053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:00.103053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:00.103053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:00.103053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:00.103957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:00.103957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:00.103957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:00.103957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:00.103957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:00.103957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:00.103957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:00.103957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:00.103957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:00.103957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:00.103957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:00.103957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:00.103957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:00.103957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:00.103957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:00.103957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:00.103957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:00.105021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:00.105021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:00.105021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:00.105021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:00.105021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 289/568] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:00.105021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d353bbb-4cd5-426d-b802-ef647d4d667f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:00.157610 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e7d51-bd83-4bd0-9a69-0b6475c55585 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:00.193966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a7557-8395-4724-b5fa-a8d4e3f6a732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:00.194788 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a7557-8395-4724-b5fa-a8d4e3f6a732 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:00.196161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a7557-8395-4724-b5fa-a8d4e3f6a732 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:00.196448 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a7557-8395-4724-b5fa-a8d4e3f6a732 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:00.205897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d353bbb-4cd5-426d-b802-ef647d4d667f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:00.206561 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a7557-8395-4724-b5fa-a8d4e3f6a732 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:00.214847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f35a7557-8395-4724-b5fa-a8d4e3f6a732 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:00.214847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:00.214847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:00.214847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:00.214847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:00.214847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:00.214847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:00.214847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:00.214847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:00.214847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:00.214847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:00.214847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:00.214847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:00.214847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:00.214847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:00.214847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:00.215776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:00.215776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:00.215776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:00.215776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:00.215776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:00.215776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:00.215776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:00.215776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:00.215776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:00.215776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:00.215776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:00.215776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:00.215776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:00.215776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:00.215776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:00.215776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:00.216625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:00.216625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:00.216625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:00.216625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:00.216625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:00.216625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:00.216625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:00.216625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:00.216625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:00.216625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:00.216625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:00.216625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:00.216625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 280/569] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:46:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:00.217386 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a7557-8395-4724-b5fa-a8d4e3f6a732 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:00.242458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226c0b7-8271-4bf3-b787-c142db643801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:00.243159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226c0b7-8271-4bf3-b787-c142db643801 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:00.243440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226c0b7-8271-4bf3-b787-c142db643801 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:00.243716 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226c0b7-8271-4bf3-b787-c142db643801 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:00.244543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226c0b7-8271-4bf3-b787-c142db643801 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:00.244856 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226c0b7-8271-4bf3-b787-c142db643801 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:00.246292 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226c0b7-8271-4bf3-b787-c142db643801 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:00.263083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 290/570] 10.222.0.70 () {56 vars in 1133 bytes} [Thu Jun 16 18:46:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:00.264287 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71c059-630f-4048-a0a5-56b89ccb2159 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:00.264575 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71c059-630f-4048-a0a5-56b89ccb2159 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:00.264850 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71c059-630f-4048-a0a5-56b89ccb2159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:00.276857 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:00.282241 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4c71c059-630f-4048-a0a5-56b89ccb2159 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:00.284399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4c71c059-630f-4048-a0a5-56b89ccb2159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:00.286389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:00.330362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a7557-8395-4724-b5fa-a8d4e3f6a732 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:00.354211 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 291/571] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:46:00.355103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71c059-630f-4048-a0a5-56b89ccb2159 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:00.363401 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab31383-2f30-4a1a-962c-11f3b4b6c6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:00.364069 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab31383-2f30-4a1a-962c-11f3b4b6c6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:00.364355 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab31383-2f30-4a1a-962c-11f3b4b6c6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:00.364630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab31383-2f30-4a1a-962c-11f3b4b6c6e6 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:00.365463 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab31383-2f30-4a1a-962c-11f3b4b6c6e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:00.365774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab31383-2f30-4a1a-962c-11f3b4b6c6e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:00.367482 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab31383-2f30-4a1a-962c-11f3b4b6c6e6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:00.385607 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 281/572] 10.222.0.70 () {56 vars in 1137 bytes} [Thu Jun 16 18:46:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:00.386423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab31383-2f30-4a1a-962c-11f3b4b6c6e6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:00.472573 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71c059-630f-4048-a0a5-56b89ccb2159 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:00.495955 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab31383-2f30-4a1a-962c-11f3b4b6c6e6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:00.505193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:00.509339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fbea9-957b-48ee-9941-24e7ca108dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:00.509998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fbea9-957b-48ee-9941-24e7ca108dce None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:00.510278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fbea9-957b-48ee-9941-24e7ca108dce None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:00.510547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fbea9-957b-48ee-9941-24e7ca108dce None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:00.511287 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fbea9-957b-48ee-9941-24e7ca108dce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:00.511587 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fbea9-957b-48ee-9941-24e7ca108dce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:00.512940 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fbea9-957b-48ee-9941-24e7ca108dce None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:00.519649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:00.526012 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 292/573] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:00.527009 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fbea9-957b-48ee-9941-24e7ca108dce None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:00.536036 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7831d7d-b637-4ed5-9b2b-61b6f61f92c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:00.536647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7831d7d-b637-4ed5-9b2b-61b6f61f92c1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:00.536931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7831d7d-b637-4ed5-9b2b-61b6f61f92c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:00.537214 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7831d7d-b637-4ed5-9b2b-61b6f61f92c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:00.537965 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7831d7d-b637-4ed5-9b2b-61b6f61f92c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:00.538272 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7831d7d-b637-4ed5-9b2b-61b6f61f92c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:00.540595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7831d7d-b637-4ed5-9b2b-61b6f61f92c1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:00.549208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 282/574] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:46:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:00.550148 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7831d7d-b637-4ed5-9b2b-61b6f61f92c1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:00.630672 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fbea9-957b-48ee-9941-24e7ca108dce None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:00.661321 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7831d7d-b637-4ed5-9b2b-61b6f61f92c1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:00.668164 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36926f-e4af-4382-8933-500424bd02c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:00.668968 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36926f-e4af-4382-8933-500424bd02c5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:00.669250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36926f-e4af-4382-8933-500424bd02c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:00.669527 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36926f-e4af-4382-8933-500424bd02c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:00.681520 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36926f-e4af-4382-8933-500424bd02c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:00.683003 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36926f-e4af-4382-8933-500424bd02c5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:00.696179 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b2ab2-86c6-48c7-802e-b48871a4c08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:00.696973 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b2ab2-86c6-48c7-802e-b48871a4c08a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:00.697291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b2ab2-86c6-48c7-802e-b48871a4c08a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:00.697567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b2ab2-86c6-48c7-802e-b48871a4c08a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:00.700630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ad36926f-e4af-4382-8933-500424bd02c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:00.700630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:00.700630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:00.700630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:00.700630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:00.700630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:00.700630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:00.700630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:00.700630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:00.700630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:00.700630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:00.700630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:00.700630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:00.700630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:00.700630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:00.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:00.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:00.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:00.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:00.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:00.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:00.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:00.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:00.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:00.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:00.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:00.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:00.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:00.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:00.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:00.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:00.701730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:00.702932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:00.702932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:00.702932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:00.702932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:00.702932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 293/575] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:00.703314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36926f-e4af-4382-8933-500424bd02c5 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:00.712759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b2ab2-86c6-48c7-802e-b48871a4c08a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:00.720549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-2a5b2ab2-86c6-48c7-802e-b48871a4c08a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:00.720549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:00.720549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:00.720549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:00.720549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:00.720549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:00.720549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:00.720549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:00.720549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:00.720549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:00.720549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:00.720549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:00.720549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:00.720549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:00.720549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:00.720549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:00.721595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:00.721595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:00.721595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:00.721595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:00.721595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:00.721595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:00.721595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:00.721595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:00.721595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:00.721595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:00.721595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:00.721595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:00.721595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:00.721595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:00.721595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:00.721595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:00.722446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:00.722446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:00.722446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:00.722446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:00.722446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:00.722446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:00.722446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:00.722446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:00.722446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:00.722446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:00.722446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:00.722446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:00.722446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 283/576] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:46:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:00.723833 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b2ab2-86c6-48c7-802e-b48871a4c08a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:00.752047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:00.756302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:00.802723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36926f-e4af-4382-8933-500424bd02c5 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:00.828037 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b2ab2-86c6-48c7-802e-b48871a4c08a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:00.837159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ea005-099e-4407-9992-c735d1d8ff44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:00.837947 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ea005-099e-4407-9992-c735d1d8ff44 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:00.838243 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ea005-099e-4407-9992-c735d1d8ff44 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:00.838515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ea005-099e-4407-9992-c735d1d8ff44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:00.848903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ea005-099e-4407-9992-c735d1d8ff44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:00.850371 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ea005-099e-4407-9992-c735d1d8ff44 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:00.862208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af2ec5-35c3-4863-8b06-b1e31216397d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:00.863097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af2ec5-35c3-4863-8b06-b1e31216397d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:00.863387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af2ec5-35c3-4863-8b06-b1e31216397d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:00.863665 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af2ec5-35c3-4863-8b06-b1e31216397d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:00.868226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-240ea005-099e-4407-9992-c735d1d8ff44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:00.868226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:00.868226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:00.868226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:00.868226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:00.868226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:00.868226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:00.868226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:00.868226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:00.868226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:00.868226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:00.868226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:00.868226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:00.868226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:00.868226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:00.869271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:00.869271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:00.869271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:00.869271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:00.869271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:00.869271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:00.869271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:00.869271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:00.869271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:00.869271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:00.869271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:00.869271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:00.869271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:00.869271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:00.869271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:00.869271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:00.869271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:00.870239 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:00.870239 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:00.870239 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:00.870239 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:00.870893 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af2ec5-35c3-4863-8b06-b1e31216397d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:00.871013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 294/577] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:00.871719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ea005-099e-4407-9992-c735d1d8ff44 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:00.872339 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af2ec5-35c3-4863-8b06-b1e31216397d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:00.886962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c6af2ec5-35c3-4863-8b06-b1e31216397d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:00.886962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:00.886962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:00.886962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:00.886962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:00.886962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:00.886962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:00.886962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:00.886962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:00.886962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:00.886962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:00.886962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:00.886962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:00.886962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:00.886962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:00.888048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:00.888048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:00.888048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:00.888048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:00.888048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:00.888048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:00.888048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:00.888048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:00.888048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:00.888048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:00.888048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:00.888048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:00.888048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:00.888048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:00.888048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:00.888048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:00.888048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:00.889183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:00.889183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:00.889183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:00.889183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:00.889183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:00.889183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:00.889183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:00.889183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:00.889183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 284/578] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:00.889183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af2ec5-35c3-4863-8b06-b1e31216397d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:00.988465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:00.989393 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ea005-099e-4407-9992-c735d1d8ff44 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:00.991933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:00.999300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af2ec5-35c3-4863-8b06-b1e31216397d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:01.026804 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd781f-a275-4acc-97ea-d96ea713691c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:01.027600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd781f-a275-4acc-97ea-d96ea713691c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:01.027884 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd781f-a275-4acc-97ea-d96ea713691c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:01.028147 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd781f-a275-4acc-97ea-d96ea713691c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:01.034563 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43527515-ba20-4dc2-9720-f6fbef56990a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:01.035259 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43527515-ba20-4dc2-9720-f6fbef56990a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:01.035542 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43527515-ba20-4dc2-9720-f6fbef56990a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:01.035822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43527515-ba20-4dc2-9720-f6fbef56990a None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:01.036544 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43527515-ba20-4dc2-9720-f6fbef56990a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:01.036850 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43527515-ba20-4dc2-9720-f6fbef56990a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:01.038206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43527515-ba20-4dc2-9720-f6fbef56990a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:01.038454 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd781f-a275-4acc-97ea-d96ea713691c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:01.040582 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd781f-a275-4acc-97ea-d96ea713691c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:01.050492 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 285/579] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:01.051309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43527515-ba20-4dc2-9720-f6fbef56990a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:01.057591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-eecd781f-a275-4acc-97ea-d96ea713691c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:01.057591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:01.057591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:01.057591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:01.057591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:01.057591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:01.057591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:01.057591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:01.057591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:01.057591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:01.057591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:01.057591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:01.057591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:01.057591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:01.057591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:01.058557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:01.058557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:01.058557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:01.058557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:01.058557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:01.058557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:01.058557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:01.058557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:01.058557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:01.058557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:01.058557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:01.058557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:01.058557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:01.058557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:01.058557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:01.058557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:01.058557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:01.059478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:01.059478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:01.059478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:01.059478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:01.059478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 295/580] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:01.063464 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd781f-a275-4acc-97ea-d96ea713691c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:01.154804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43527515-ba20-4dc2-9720-f6fbef56990a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:01.166684 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd781f-a275-4acc-97ea-d96ea713691c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:01.188234 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c7dc8-1752-4b7b-8ad2-278fd01be29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:01.188900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c7dc8-1752-4b7b-8ad2-278fd01be29a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:01.189185 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c7dc8-1752-4b7b-8ad2-278fd01be29a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:01.189470 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c7dc8-1752-4b7b-8ad2-278fd01be29a None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:01.190210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c7dc8-1752-4b7b-8ad2-278fd01be29a None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:01.190522 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c7dc8-1752-4b7b-8ad2-278fd01be29a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:01.191900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c7dc8-1752-4b7b-8ad2-278fd01be29a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:01.200442 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234908a-eb46-4ce4-acaa-803b21030b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:01.200591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 286/581] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:46:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:01.201107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234908a-eb46-4ce4-acaa-803b21030b8a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:01.201211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c7dc8-1752-4b7b-8ad2-278fd01be29a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:01.201383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234908a-eb46-4ce4-acaa-803b21030b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:01.201675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234908a-eb46-4ce4-acaa-803b21030b8a None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:01.202362 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234908a-eb46-4ce4-acaa-803b21030b8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:01.202695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234908a-eb46-4ce4-acaa-803b21030b8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:01.204082 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234908a-eb46-4ce4-acaa-803b21030b8a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:01.216471 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:01.216917 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 296/582] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:01.217639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234908a-eb46-4ce4-acaa-803b21030b8a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:01.233366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:01.307158 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c7dc8-1752-4b7b-8ad2-278fd01be29a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:01.327448 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234908a-eb46-4ce4-acaa-803b21030b8a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:01.342847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd0915-9322-4543-8426-456d1522b599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:01.343660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd0915-9322-4543-8426-456d1522b599 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:01.344002 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd0915-9322-4543-8426-456d1522b599 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:01.344282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd0915-9322-4543-8426-456d1522b599 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:01.345097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd0915-9322-4543-8426-456d1522b599 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:01.345412 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd0915-9322-4543-8426-456d1522b599 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:01.346855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd0915-9322-4543-8426-456d1522b599 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:01.358897 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 287/583] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:01.359885 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd0915-9322-4543-8426-456d1522b599 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:01.361975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954d781-0710-4a78-a24f-e315bfaa2739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:01.362792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954d781-0710-4a78-a24f-e315bfaa2739 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:01.363076 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954d781-0710-4a78-a24f-e315bfaa2739 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:01.363352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954d781-0710-4a78-a24f-e315bfaa2739 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:01.370486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954d781-0710-4a78-a24f-e315bfaa2739 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:01.371965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954d781-0710-4a78-a24f-e315bfaa2739 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:01.386406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-2954d781-0710-4a78-a24f-e315bfaa2739 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:01.386406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:01.386406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:01.386406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:01.386406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:01.386406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:01.386406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:01.386406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:01.386406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:01.386406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:01.386406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:01.386406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:01.386406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:01.386406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:01.386406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:01.387525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:01.387525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:01.387525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:01.387525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:01.387525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:01.387525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:01.387525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:01.387525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:01.387525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:01.387525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:01.387525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:01.387525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:01.387525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:01.387525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:01.387525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:01.387525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:01.387525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:01.388625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:01.388625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:01.388625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:01.388625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:01.388625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:01.388625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:01.388625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:01.388625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:01.388625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 297/584] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:01.388625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954d781-0710-4a78-a24f-e315bfaa2739 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:01.436264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:01.464735 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:01.466362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd0915-9322-4543-8426-456d1522b599 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:01.491139 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954d781-0710-4a78-a24f-e315bfaa2739 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:01.500407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88c2d4-7f64-4bcd-948d-a2c2609659f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:01.501069 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88c2d4-7f64-4bcd-948d-a2c2609659f8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:01.501356 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88c2d4-7f64-4bcd-948d-a2c2609659f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:01.501635 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88c2d4-7f64-4bcd-948d-a2c2609659f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:01.502350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88c2d4-7f64-4bcd-948d-a2c2609659f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:01.502680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88c2d4-7f64-4bcd-948d-a2c2609659f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:01.504040 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88c2d4-7f64-4bcd-948d-a2c2609659f8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:01.516101 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 288/585] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:01.521700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88c2d4-7f64-4bcd-948d-a2c2609659f8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:01.525479 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2baf8fd-0559-48ba-ba2c-6b8456df461b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:01.526268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2baf8fd-0559-48ba-ba2c-6b8456df461b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:01.526546 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2baf8fd-0559-48ba-ba2c-6b8456df461b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:01.526846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2baf8fd-0559-48ba-ba2c-6b8456df461b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:01.538042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2baf8fd-0559-48ba-ba2c-6b8456df461b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:01.539632 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2baf8fd-0559-48ba-ba2c-6b8456df461b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:01.556765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-d2baf8fd-0559-48ba-ba2c-6b8456df461b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:01.556765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:01.556765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:01.556765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:01.556765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:01.556765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:01.556765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:01.556765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:01.556765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:01.556765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:01.556765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:01.556765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:01.556765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:01.556765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:01.556765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:01.557781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:01.557781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:01.557781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:01.557781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:01.557781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:01.557781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:01.557781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:01.557781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:01.557781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:01.557781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:01.557781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:01.557781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:01.557781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:01.557781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:01.557781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:01.557781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:01.557781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:01.558934 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:01.558934 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:01.558934 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:01.558934 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:01.558934 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 298/586] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:01.558934 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2baf8fd-0559-48ba-ba2c-6b8456df461b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:01.624062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88c2d4-7f64-4bcd-948d-a2c2609659f8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:01.655695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:01.657079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2baf8fd-0559-48ba-ba2c-6b8456df461b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:01.662769 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574369a8-20c3-469b-ad6f-d116754f87bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:01.664323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574369a8-20c3-469b-ad6f-d116754f87bb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:01.664605 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574369a8-20c3-469b-ad6f-d116754f87bb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:01.664985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574369a8-20c3-469b-ad6f-d116754f87bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:01.675122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574369a8-20c3-469b-ad6f-d116754f87bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:01.676687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574369a8-20c3-469b-ad6f-d116754f87bb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:01.686300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:01.693974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-574369a8-20c3-469b-ad6f-d116754f87bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:01.693974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:01.693974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:01.693974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:01.693974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:01.693974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:01.693974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:01.693974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:01.693974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:01.693974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:01.693974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:01.693974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:01.693974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:01.693974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:01.693974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:01.695393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:01.695393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:01.695393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:01.695393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:01.695393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:01.695393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:01.695393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:01.695393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:01.695393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:01.695393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:01.695393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:01.695393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:01.695393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:01.695393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:01.695393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:01.695393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:01.695393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:01.696919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:01.696919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:01.696919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:01.696919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:01.696919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:01.696919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:01.696919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:01.696919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:01.696919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 289/587] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:01.696919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574369a8-20c3-469b-ad6f-d116754f87bb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:01.700756 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a260d49-419d-4e15-bb63-9f1552641e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:01.701513 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a260d49-419d-4e15-bb63-9f1552641e6f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:01.701799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a260d49-419d-4e15-bb63-9f1552641e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:01.702072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a260d49-419d-4e15-bb63-9f1552641e6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:01.702932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a260d49-419d-4e15-bb63-9f1552641e6f None admin] RBAC: Adding query filter params (role.id=c123034c11cc4d9ea6cab00e05bdcd53, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=7b22fe0ab0354385924923d0d443acae) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:01.703244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a260d49-419d-4e15-bb63-9f1552641e6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:01.704673 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a260d49-419d-4e15-bb63-9f1552641e6f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:01.711063 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 299/588] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:46:01 2022] GET /identity/v3/role_assignments?user.id=7b22fe0ab0354385924923d0d443acae&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=c123034c11cc4d9ea6cab00e05bdcd53 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:01.712022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a260d49-419d-4e15-bb63-9f1552641e6f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:01.817888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574369a8-20c3-469b-ad6f-d116754f87bb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:01.832516 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a260d49-419d-4e15-bb63-9f1552641e6f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:01.857465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb2db7-87fd-4639-8fbc-8e8a830d59b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:01.858141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb2db7-87fd-4639-8fbc-8e8a830d59b6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:01.858424 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb2db7-87fd-4639-8fbc-8e8a830d59b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:01.858724 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb2db7-87fd-4639-8fbc-8e8a830d59b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:01.859524 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb2db7-87fd-4639-8fbc-8e8a830d59b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:01.859830 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb2db7-87fd-4639-8fbc-8e8a830d59b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:01.861339 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb2db7-87fd-4639-8fbc-8e8a830d59b6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:01.872312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0be1c7-02b5-4e83-ac9f-ed135d5cbf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:01.873234 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0be1c7-02b5-4e83-ac9f-ed135d5cbf8d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:01.873511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0be1c7-02b5-4e83-ac9f-ed135d5cbf8d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:01.873790 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0be1c7-02b5-4e83-ac9f-ed135d5cbf8d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:01.876998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 290/589] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:01.877703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb2db7-87fd-4639-8fbc-8e8a830d59b6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:01.882087 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0be1c7-02b5-4e83-ac9f-ed135d5cbf8d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:01.884720 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0be1c7-02b5-4e83-ac9f-ed135d5cbf8d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:01.900481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-bc0be1c7-02b5-4e83-ac9f-ed135d5cbf8d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:46:01.900481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:01.900481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:01.900481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:01.900481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:01.900481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:01.900481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:01.900481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:01.900481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:01.900481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:01.900481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:01.900481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:01.900481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:01.900481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:01.900481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:01.900481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:01.901485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:01.902433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:01.902433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:01.902433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:01.902433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:01.902433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:01.902433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:01.902433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:01.902433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:01.902433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:46:01.902433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:01.902433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 300/590] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:46:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:01.902433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0be1c7-02b5-4e83-ac9f-ed135d5cbf8d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:01.956299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:01.962315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:02.014249 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0be1c7-02b5-4e83-ac9f-ed135d5cbf8d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:02.017245 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb2db7-87fd-4639-8fbc-8e8a830d59b6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:02.052336 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8027a-91ff-4fa1-a402-3d9c43bde512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:02.053107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8027a-91ff-4fa1-a402-3d9c43bde512 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:02.053460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8027a-91ff-4fa1-a402-3d9c43bde512 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:02.053797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8027a-91ff-4fa1-a402-3d9c43bde512 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:02.054719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8027a-91ff-4fa1-a402-3d9c43bde512 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:02.055091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8027a-91ff-4fa1-a402-3d9c43bde512 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:02.056569 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8027a-91ff-4fa1-a402-3d9c43bde512 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:02.057037 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71b56e-0942-44f0-86fe-9b1566ef8136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:02.057875 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71b56e-0942-44f0-86fe-9b1566ef8136 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:02.058300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71b56e-0942-44f0-86fe-9b1566ef8136 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:02.058750 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71b56e-0942-44f0-86fe-9b1566ef8136 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:02.059615 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71b56e-0942-44f0-86fe-9b1566ef8136 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:02.060061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71b56e-0942-44f0-86fe-9b1566ef8136 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:02.061565 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71b56e-0942-44f0-86fe-9b1566ef8136 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:02.069661 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 301/591] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:02.070981 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8027a-91ff-4fa1-a402-3d9c43bde512 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:02.075508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 291/592] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:02.076622 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71b56e-0942-44f0-86fe-9b1566ef8136 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:02.183539 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8027a-91ff-4fa1-a402-3d9c43bde512 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:02.185059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71b56e-0942-44f0-86fe-9b1566ef8136 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:02.197042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:02.202148 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:02.224095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19df66-9d3a-4c5b-905a-021a3298a124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:02.225093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19df66-9d3a-4c5b-905a-021a3298a124 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:02.225490 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19df66-9d3a-4c5b-905a-021a3298a124 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:02.225818 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19df66-9d3a-4c5b-905a-021a3298a124 None admin] PATH_INFO: `/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/8342da838fe4462b9de2817ad3ac4ff7/roles/c123034c11cc4d9ea6cab00e05bdcd53` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:02.226532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c3a7b-cc18-4737-a796-5b5bf7b0d9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:02.227230 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c3a7b-cc18-4737-a796-5b5bf7b0d9da None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:02.227513 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c3a7b-cc18-4737-a796-5b5bf7b0d9da None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:02.227787 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c3a7b-cc18-4737-a796-5b5bf7b0d9da None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:02.228622 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c3a7b-cc18-4737-a796-5b5bf7b0d9da None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:02.228933 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c3a7b-cc18-4737-a796-5b5bf7b0d9da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:02.230348 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c3a7b-cc18-4737-a796-5b5bf7b0d9da None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:02.242334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19df66-9d3a-4c5b-905a-021a3298a124 None admin] RBAC: Authorizing `identity:create_grant(project_id=34185125e75d4aad8ad5382ab22ee8a2, user_id=8342da838fe4462b9de2817ad3ac4ff7, role_id=c123034c11cc4d9ea6cab00e05bdcd53)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:02.244148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19df66-9d3a-4c5b-905a-021a3298a124 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:02.250752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 292/593] 10.222.0.70 () {56 vars in 1139 bytes} [Thu Jun 16 18:46:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:02.250752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c3a7b-cc18-4737-a796-5b5bf7b0d9da None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:02.266606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 302/594] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:46:02 2022] PUT /identity/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/8342da838fe4462b9de2817ad3ac4ff7/roles/c123034c11cc4d9ea6cab00e05bdcd53 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:46:02.266606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19df66-9d3a-4c5b-905a-021a3298a124 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:02.387704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c3a7b-cc18-4737-a796-5b5bf7b0d9da None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:02.415275 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19df66-9d3a-4c5b-905a-021a3298a124 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:02.437771 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d3926-a70d-4922-a946-a294107fc829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:02.439080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d3926-a70d-4922-a946-a294107fc829 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:02.439386 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d3926-a70d-4922-a946-a294107fc829 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:02.439664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d3926-a70d-4922-a946-a294107fc829 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:02.450502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079d3926-a70d-4922-a946-a294107fc829 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:02.453775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079d3926-a70d-4922-a946-a294107fc829 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:02.466406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da114a1-0008-4739-bd74-347db1905f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:02.467694 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da114a1-0008-4739-bd74-347db1905f21 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:02.467977 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da114a1-0008-4739-bd74-347db1905f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:02.468247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da114a1-0008-4739-bd74-347db1905f21 None admin] PATH_INFO: `/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/8563a25d16134d639a2a8849976fed9a/roles/c123034c11cc4d9ea6cab00e05bdcd53` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:02.471408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-079d3926-a70d-4922-a946-a294107fc829 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:46:02.471408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:02.471408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:02.471408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:02.471408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:02.471408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:02.471408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:02.471408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:02.471408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:02.471408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:02.471408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:02.471408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:02.471408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:02.471408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:02.471408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:02.471408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:02.472332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:02.473303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:02.473303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:02.473303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:02.473303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:02.473303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:02.473303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:02.473303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:02.473303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:02.473303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:46:02.473303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:02.473303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 293/595] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:46:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:02.473934 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d3926-a70d-4922-a946-a294107fc829 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:02.489236 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da114a1-0008-4739-bd74-347db1905f21 None admin] RBAC: Authorizing `identity:create_grant(project_id=34185125e75d4aad8ad5382ab22ee8a2, user_id=8563a25d16134d639a2a8849976fed9a, role_id=c123034c11cc4d9ea6cab00e05bdcd53)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:02.490775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da114a1-0008-4739-bd74-347db1905f21 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:02.507090 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 303/596] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:46:02 2022] PUT /identity/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/8563a25d16134d639a2a8849976fed9a/roles/c123034c11cc4d9ea6cab00e05bdcd53 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:46:02.513943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da114a1-0008-4739-bd74-347db1905f21 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:02.536647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:02.544922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:02.682209 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d3926-a70d-4922-a946-a294107fc829 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:02.702220 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da114a1-0008-4739-bd74-347db1905f21 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:02.737748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685138f-d6df-425f-87d8-33b207a19f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:02.738773 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685138f-d6df-425f-87d8-33b207a19f70 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:02.739041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685138f-d6df-425f-87d8-33b207a19f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:02.739317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685138f-d6df-425f-87d8-33b207a19f70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:02.750746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685138f-d6df-425f-87d8-33b207a19f70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:02.752196 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685138f-d6df-425f-87d8-33b207a19f70 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:02.756320 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b2535-90f7-4b42-aeb9-ea1b6d9066f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:02.757007 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b2535-90f7-4b42-aeb9-ea1b6d9066f1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:02.758023 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b2535-90f7-4b42-aeb9-ea1b6d9066f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:02.758817 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b2535-90f7-4b42-aeb9-ea1b6d9066f1 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:02.759269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b2535-90f7-4b42-aeb9-ea1b6d9066f1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:02.759637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b2535-90f7-4b42-aeb9-ea1b6d9066f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:02.761102 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b2535-90f7-4b42-aeb9-ea1b6d9066f1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:02.774650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-d685138f-d6df-425f-87d8-33b207a19f70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:02.774650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:02.774650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:02.774650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:02.774650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:02.774650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:02.774650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:02.774650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:02.774650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:02.774650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:02.774650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:02.774650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:02.774650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:02.774650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:02.774650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:02.775634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:02.775634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:02.775634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:02.775634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:02.775634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:02.775634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:02.775634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:02.775634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:02.775634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:02.775634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:02.775634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:02.775634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:02.775634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:02.775634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:02.775634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:02.775634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:02.775634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:02.776549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:02.776549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:02.776549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:02.776549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:02.776549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 294/597] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:02.782412 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685138f-d6df-425f-87d8-33b207a19f70 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:02.792600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 304/598] 10.222.0.70 () {56 vars in 1137 bytes} [Thu Jun 16 18:46:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:02.798176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b2535-90f7-4b42-aeb9-ea1b6d9066f1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:02.897069 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685138f-d6df-425f-87d8-33b207a19f70 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:02.916893 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:02.951342 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864ab2e-9db6-449e-a4aa-69787a7392ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:02.952024 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ab2e-9db6-449e-a4aa-69787a7392ab None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:02.952364 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ab2e-9db6-449e-a4aa-69787a7392ab None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:02.952583 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ab2e-9db6-449e-a4aa-69787a7392ab None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:02.953322 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864ab2e-9db6-449e-a4aa-69787a7392ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:02.953622 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864ab2e-9db6-449e-a4aa-69787a7392ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:02.955012 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864ab2e-9db6-449e-a4aa-69787a7392ab None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:02.959756 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:02.984632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 295/599] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:02.990017 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864ab2e-9db6-449e-a4aa-69787a7392ab None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:03.030525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b2535-90f7-4b42-aeb9-ea1b6d9066f1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:03.095347 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb1542-14d5-4256-9d64-9b6d3eb9ee62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:03.096164 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb1542-14d5-4256-9d64-9b6d3eb9ee62 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:03.096438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb1542-14d5-4256-9d64-9b6d3eb9ee62 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:03.096710 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb1542-14d5-4256-9d64-9b6d3eb9ee62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:03.119920 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb1542-14d5-4256-9d64-9b6d3eb9ee62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:03.121626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb1542-14d5-4256-9d64-9b6d3eb9ee62 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:03.137939 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864ab2e-9db6-449e-a4aa-69787a7392ab None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:03.152718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-5bdb1542-14d5-4256-9d64-9b6d3eb9ee62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:03.152718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:03.152718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:03.152718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:03.152718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:03.152718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:03.152718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:03.152718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:03.152718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:03.152718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:03.152718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:03.152718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:03.152718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:03.152718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:03.152718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:03.153944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:03.153944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:03.153944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:03.153944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:03.153944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:03.153944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:03.153944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:03.153944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:03.153944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:03.153944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:03.153944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:03.153944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:03.153944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:03.153944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:03.153944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:03.153944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:03.153944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:03.155135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:03.155135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:03.155135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:03.155135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:03.155408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 305/600] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:03.179810 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb1542-14d5-4256-9d64-9b6d3eb9ee62 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:03.184262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb5df2-2997-4b29-849c-13715a2cf223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:03.188512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb5df2-2997-4b29-849c-13715a2cf223 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:03.188512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb5df2-2997-4b29-849c-13715a2cf223 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:03.188512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb5df2-2997-4b29-849c-13715a2cf223 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:03.201826 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb5df2-2997-4b29-849c-13715a2cf223 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:03.205173 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb5df2-2997-4b29-849c-13715a2cf223 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:03.234211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-9abb5df2-2997-4b29-849c-13715a2cf223 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:03.234211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:03.234211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:03.234211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:03.234211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:03.234211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:03.234211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:03.234211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:03.234211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:03.234211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:03.234211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:03.234211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:03.234211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:03.234211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:03.234211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:03.236294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:03.236294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:03.236294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:03.236294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:03.236294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:03.236294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:03.236294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:03.236294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:03.236294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:03.236294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:03.236294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:03.236294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:03.236294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:03.236294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:03.236294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:03.236294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:03.236294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:03.238442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:03.238442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:03.238442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:03.238442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:03.238442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:03.238442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:03.238442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:03.238442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:03.239576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 296/601] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:03.244933 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb5df2-2997-4b29-849c-13715a2cf223 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:03.248102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:03.352642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb5df2-2997-4b29-849c-13715a2cf223 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:03.387156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb1542-14d5-4256-9d64-9b6d3eb9ee62 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:03.389570 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd196adb-debb-4e87-aeb2-5067be9ea8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:03.390313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd196adb-debb-4e87-aeb2-5067be9ea8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:03.390658 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd196adb-debb-4e87-aeb2-5067be9ea8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:03.391129 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd196adb-debb-4e87-aeb2-5067be9ea8c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:03.392731 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd196adb-debb-4e87-aeb2-5067be9ea8c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:03.393551 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd196adb-debb-4e87-aeb2-5067be9ea8c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:03.396452 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd196adb-debb-4e87-aeb2-5067be9ea8c3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:03.425744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 297/602] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:03.434637 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd196adb-debb-4e87-aeb2-5067be9ea8c3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:03.438966 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:03.508505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205261c-d795-4aa6-9204-cb4bce67771a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:03.508712 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205261c-d795-4aa6-9204-cb4bce67771a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:03.508930 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205261c-d795-4aa6-9204-cb4bce67771a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:03.509202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205261c-d795-4aa6-9204-cb4bce67771a None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:03.509625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:03.509953 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205261c-d795-4aa6-9204-cb4bce67771a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:03.510268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205261c-d795-4aa6-9204-cb4bce67771a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:03.511838 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205261c-d795-4aa6-9204-cb4bce67771a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:03.544994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd196adb-debb-4e87-aeb2-5067be9ea8c3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:03.549056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 306/603] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:03.554449 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205261c-d795-4aa6-9204-cb4bce67771a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:03.584356 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4159dae-beac-4472-9538-1685245a1435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:03.585630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4159dae-beac-4472-9538-1685245a1435 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:03.586112 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4159dae-beac-4472-9538-1685245a1435 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:03.586606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4159dae-beac-4472-9538-1685245a1435 None admin] PATH_INFO: `/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/02a83f6e7aa448d6984a78914c1748cf/roles/c123034c11cc4d9ea6cab00e05bdcd53` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:03.602921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4159dae-beac-4472-9538-1685245a1435 None admin] RBAC: Authorizing `identity:create_grant(project_id=34185125e75d4aad8ad5382ab22ee8a2, user_id=02a83f6e7aa448d6984a78914c1748cf, role_id=c123034c11cc4d9ea6cab00e05bdcd53)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:03.604730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4159dae-beac-4472-9538-1685245a1435 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:03.622839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 298/604] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:46:03 2022] PUT /identity/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/02a83f6e7aa448d6984a78914c1748cf/roles/c123034c11cc4d9ea6cab00e05bdcd53 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:46:03.714107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205261c-d795-4aa6-9204-cb4bce67771a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:03.755399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74812e69-5d6a-4816-ba9b-467a1393de04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:03.756563 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74812e69-5d6a-4816-ba9b-467a1393de04 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:03.757042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74812e69-5d6a-4816-ba9b-467a1393de04 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:03.757751 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74812e69-5d6a-4816-ba9b-467a1393de04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:03.765825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74812e69-5d6a-4816-ba9b-467a1393de04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:03.767784 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74812e69-5d6a-4816-ba9b-467a1393de04 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:03.784654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-74812e69-5d6a-4816-ba9b-467a1393de04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:03.784654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:03.784654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:03.784654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:03.784654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:03.784654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:03.784654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:03.784654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:03.784654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:03.784654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:03.784654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:03.784654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:03.784654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:03.784654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:03.784654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:03.785663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:03.785663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:03.785663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:03.785663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:03.785663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:03.785663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:03.785663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:03.785663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:03.785663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:03.785663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:03.785663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:03.785663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:03.785663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:03.785663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:03.785663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:03.785663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:03.785663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:03.786596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:03.786596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:03.786596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:03.786596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:03.786596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:03.786596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:03.786596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:03.786596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:03.789605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 307/605] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:03.796797 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4159dae-beac-4472-9538-1685245a1435 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:03.916609 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4159dae-beac-4472-9538-1685245a1435 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:03.934988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:03.957618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44913d12-5208-410f-9db9-4a96303e18bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:03.957618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44913d12-5208-410f-9db9-4a96303e18bc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:03.957618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44913d12-5208-410f-9db9-4a96303e18bc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:03.957618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44913d12-5208-410f-9db9-4a96303e18bc None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:03.957995 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44913d12-5208-410f-9db9-4a96303e18bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:03.957995 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44913d12-5208-410f-9db9-4a96303e18bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:03.959185 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44913d12-5208-410f-9db9-4a96303e18bc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:03.974456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 299/606] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:03.980513 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74812e69-5d6a-4816-ba9b-467a1393de04 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:04.025932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:04.087703 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74812e69-5d6a-4816-ba9b-467a1393de04 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:04.124073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf0382-c8b7-45a2-b6fa-e0ef61e3de4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:04.124968 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf0382-c8b7-45a2-b6fa-e0ef61e3de4b None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:04.125251 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf0382-c8b7-45a2-b6fa-e0ef61e3de4b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:04.125521 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf0382-c8b7-45a2-b6fa-e0ef61e3de4b None admin] PATH_INFO: `/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/a47de73e3f3b4979bff83140bf287275/roles/c123034c11cc4d9ea6cab00e05bdcd53` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:04.141247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf0382-c8b7-45a2-b6fa-e0ef61e3de4b None admin] RBAC: Authorizing `identity:create_grant(project_id=34185125e75d4aad8ad5382ab22ee8a2, user_id=a47de73e3f3b4979bff83140bf287275, role_id=c123034c11cc4d9ea6cab00e05bdcd53)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:04.142725 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf0382-c8b7-45a2-b6fa-e0ef61e3de4b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:04.162244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 308/607] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:46:03 2022] PUT /identity/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/a47de73e3f3b4979bff83140bf287275/roles/c123034c11cc4d9ea6cab00e05bdcd53 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:46:04.299560 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7992f-e3c4-4e00-9801-02959e1e51be None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:04.299756 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7992f-e3c4-4e00-9801-02959e1e51be None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:04.300038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7992f-e3c4-4e00-9801-02959e1e51be None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:04.301707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 300/608] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:04.307619 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fc194-3bcf-4ef3-93a3-a890e05ea2db None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:04.307915 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fc194-3bcf-4ef3-93a3-a890e05ea2db None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:04.308187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fc194-3bcf-4ef3-93a3-a890e05ea2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:04.327459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-384fc194-3bcf-4ef3-93a3-a890e05ea2db None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:04.328679 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-384fc194-3bcf-4ef3-93a3-a890e05ea2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:04.404052 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 309/609] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:46:04.432315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7992f-e3c4-4e00-9801-02959e1e51be None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:04.562464 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7992f-e3c4-4e00-9801-02959e1e51be None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:04.603812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a94c6-a44c-4e63-94ea-b7de801887fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:04.604626 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a94c6-a44c-4e63-94ea-b7de801887fa None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:04.604912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a94c6-a44c-4e63-94ea-b7de801887fa None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:04.605187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a94c6-a44c-4e63-94ea-b7de801887fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:04.615861 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a94c6-a44c-4e63-94ea-b7de801887fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:04.624455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-de7a94c6-a44c-4e63-94ea-b7de801887fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:04.624455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:04.624455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:04.624455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:04.624455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:04.624455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:04.624455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:04.624455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:04.624455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:04.624455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:04.624455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:04.624455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:04.624455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:04.624455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:04.624455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:04.624455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:04.625385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:04.625385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:04.625385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:04.625385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:04.625385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:04.625385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:04.625385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:04.625385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:04.625385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:04.625385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:04.625385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:04.625385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:04.625385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:04.625385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:04.625385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:04.625385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:04.626220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:04.626220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:04.626220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:04.626220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:04.626220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:04.626220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:04.626220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:04.626220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:04.626220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:04.626220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:04.626220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:04.626220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:04.626220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 301/610] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:46:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:04.632187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fc194-3bcf-4ef3-93a3-a890e05ea2db None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:04.655314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:04.740794 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fc194-3bcf-4ef3-93a3-a890e05ea2db None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:04.783157 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90547a08-3405-43bb-aac6-b2dca57f31af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:04.784176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90547a08-3405-43bb-aac6-b2dca57f31af None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:04.784600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90547a08-3405-43bb-aac6-b2dca57f31af None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:04.784885 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90547a08-3405-43bb-aac6-b2dca57f31af None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:04.785645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90547a08-3405-43bb-aac6-b2dca57f31af None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:04.785955 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90547a08-3405-43bb-aac6-b2dca57f31af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:04.787408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90547a08-3405-43bb-aac6-b2dca57f31af None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:04.800549 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 310/611] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:46:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:04.806160 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a94c6-a44c-4e63-94ea-b7de801887fa None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:04.845921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:04.947670 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a94c6-a44c-4e63-94ea-b7de801887fa None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:04.984566 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316718f-f100-4b79-a932-1ba5f94120fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:04.985362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316718f-f100-4b79-a932-1ba5f94120fc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:04.985642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316718f-f100-4b79-a932-1ba5f94120fc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:04.985918 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316718f-f100-4b79-a932-1ba5f94120fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:04.997085 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316718f-f100-4b79-a932-1ba5f94120fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:04.998520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316718f-f100-4b79-a932-1ba5f94120fc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:05.016261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f316718f-f100-4b79-a932-1ba5f94120fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:05.016261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:05.016261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:05.016261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:05.016261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:05.016261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:05.016261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:05.016261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:05.016261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:05.016261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:05.016261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:05.016261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:05.016261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:05.016261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:05.016261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:05.017206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:05.017206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:05.017206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:05.017206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:05.017206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:05.017206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:05.017206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:05.017206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:05.017206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:05.017206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:05.017206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:05.017206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:05.017206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:05.017206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:05.017206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:05.017206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:05.017206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:05.018381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:05.018381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:05.018381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:05.018381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:05.018381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 302/612] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:05.020684 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcf644-c54b-4954-b2b5-d43bbd6035e1 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:05.021198 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcf644-c54b-4954-b2b5-d43bbd6035e1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:05.021678 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcf644-c54b-4954-b2b5-d43bbd6035e1 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:05.027414 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 311/613] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:05.029877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316718f-f100-4b79-a932-1ba5f94120fc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:05.032365 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f90d5-b733-45a1-8570-2dfbf4693252 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:05.032865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f90d5-b733-45a1-8570-2dfbf4693252 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:05.033347 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f90d5-b733-45a1-8570-2dfbf4693252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:05.061584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e12f90d5-b733-45a1-8570-2dfbf4693252 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:05.064376 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e12f90d5-b733-45a1-8570-2dfbf4693252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:05.158732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 312/614] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:46:05.194710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:05.200884 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f90d5-b733-45a1-8570-2dfbf4693252 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:05.234962 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:05.241639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316718f-f100-4b79-a932-1ba5f94120fc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:05.278773 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7485da-797e-4ae3-bb69-a2d6254cdae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:05.279602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7485da-797e-4ae3-bb69-a2d6254cdae1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:05.279990 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7485da-797e-4ae3-bb69-a2d6254cdae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:05.280375 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7485da-797e-4ae3-bb69-a2d6254cdae1 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:05.281206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7485da-797e-4ae3-bb69-a2d6254cdae1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:05.281634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7485da-797e-4ae3-bb69-a2d6254cdae1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:05.283136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7485da-797e-4ae3-bb69-a2d6254cdae1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:05.296506 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 303/615] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:05.297922 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e826ab-21ca-4bc8-9c50-af326aa5d6f1 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:05.302609 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e826ab-21ca-4bc8-9c50-af326aa5d6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:05.303017 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e826ab-21ca-4bc8-9c50-af326aa5d6f1 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:05.309502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 304/616] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:05.309502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e826ab-21ca-4bc8-9c50-af326aa5d6f1 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:05.350603 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f90d5-b733-45a1-8570-2dfbf4693252 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:05.391319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe300b49-0f80-42c1-897a-59b68e030248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:05.392154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe300b49-0f80-42c1-897a-59b68e030248 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:05.392440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe300b49-0f80-42c1-897a-59b68e030248 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:05.392718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe300b49-0f80-42c1-897a-59b68e030248 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:05.404552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe300b49-0f80-42c1-897a-59b68e030248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:05.412732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-fe300b49-0f80-42c1-897a-59b68e030248 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:05.412732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:05.412732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:05.412732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:05.412732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:05.412732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:05.412732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:05.412732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:05.412732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:05.412732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:05.412732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:05.412732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:05.412732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:05.412732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:05.412732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:05.412732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:05.413866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:05.413866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:05.413866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:05.413866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:05.413866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:05.413866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:05.413866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:05.413866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:05.413866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:05.413866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:05.413866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:05.413866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:05.413866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:05.413866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:05.413866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:05.413866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:05.414757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:05.414757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:05.414757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:05.414757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:05.414757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:05.414757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:05.414757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:05.414757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:05.414757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:05.414757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:05.414757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:05.414757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:05.418960 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 313/617] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:46:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:05.420298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade44cd-8012-4e1c-930e-1d26a196c70b None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:05.420592 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade44cd-8012-4e1c-930e-1d26a196c70b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:05.420878 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade44cd-8012-4e1c-930e-1d26a196c70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:05.424533 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e826ab-21ca-4bc8-9c50-af326aa5d6f1 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:05.439047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4ade44cd-8012-4e1c-930e-1d26a196c70b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:05.440244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4ade44cd-8012-4e1c-930e-1d26a196c70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:05.469438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b9368-3e4d-4e54-9c14-fa099d161a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:05.471293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b9368-3e4d-4e54-9c14-fa099d161a21 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:05.471752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b9368-3e4d-4e54-9c14-fa099d161a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:05.472454 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b9368-3e4d-4e54-9c14-fa099d161a21 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:05.481594 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b9368-3e4d-4e54-9c14-fa099d161a21 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:05.483091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b9368-3e4d-4e54-9c14-fa099d161a21 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:05.499211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-609b9368-3e4d-4e54-9c14-fa099d161a21 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:46:05.499211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:05.499211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:05.499211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:05.499211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:05.499211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:05.499211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:05.499211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:05.499211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:05.499211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:05.499211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:05.499211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:05.499211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:05.499211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:05.499211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:05.499211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:05.500210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:05.501167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:05.501167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:05.501167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:05.501167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:05.501167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:05.501167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:05.501167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:05.501167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:05.501167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:46:05.501167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:05.504020 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 305/618] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:46:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:05.505153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b9368-3e4d-4e54-9c14-fa099d161a21 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:05.509708 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:05.525132 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 314/619] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:46:05.525132 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade44cd-8012-4e1c-930e-1d26a196c70b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:05.528617 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:05.637003 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade44cd-8012-4e1c-930e-1d26a196c70b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:05.646200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b9368-3e4d-4e54-9c14-fa099d161a21 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:05.674683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb8065-b2e0-44e9-81c7-2713e264a155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:05.675440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb8065-b2e0-44e9-81c7-2713e264a155 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:05.675977 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb8065-b2e0-44e9-81c7-2713e264a155 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:05.676261 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb8065-b2e0-44e9-81c7-2713e264a155 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:05.677103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb8065-b2e0-44e9-81c7-2713e264a155 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:05.677434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb8065-b2e0-44e9-81c7-2713e264a155 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:05.679276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb8065-b2e0-44e9-81c7-2713e264a155 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:05.696908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc8832-ce99-432f-86c4-513be978b55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:05.697798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc8832-ce99-432f-86c4-513be978b55d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:05.698218 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc8832-ce99-432f-86c4-513be978b55d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:05.698507 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc8832-ce99-432f-86c4-513be978b55d None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:05.699346 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc8832-ce99-432f-86c4-513be978b55d None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:05.699660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc8832-ce99-432f-86c4-513be978b55d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:05.700535 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 315/620] 10.222.0.70 () {56 vars in 1135 bytes} [Thu Jun 16 18:46:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:05.701215 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc8832-ce99-432f-86c4-513be978b55d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:05.701528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb8065-b2e0-44e9-81c7-2713e264a155 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:05.714657 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 306/621] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:46:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:05.715559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc8832-ce99-432f-86c4-513be978b55d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:05.773188 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:05.811642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:05.823006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb8065-b2e0-44e9-81c7-2713e264a155 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:05.859537 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd352ac-eefe-4595-99f7-4c2c85246b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:05.860332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd352ac-eefe-4595-99f7-4c2c85246b8c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:05.860612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd352ac-eefe-4595-99f7-4c2c85246b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:05.860885 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd352ac-eefe-4595-99f7-4c2c85246b8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:05.865414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc8832-ce99-432f-86c4-513be978b55d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:05.873406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd352ac-eefe-4595-99f7-4c2c85246b8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:05.881071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-bdd352ac-eefe-4595-99f7-4c2c85246b8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:05.881071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:05.881071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:05.881071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:05.881071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:05.881071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:05.881071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:05.881071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:05.881071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:05.881071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:05.881071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:05.881071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:05.881071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:05.881071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:05.881071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:05.881071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:05.882554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:05.882554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:05.882554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:05.882554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:05.882554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:05.882554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:05.882554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:05.882554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:05.882554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:05.882554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:05.882554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:05.882554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:05.882554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:05.882554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:05.882554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:05.882554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:05.883424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:05.883424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:05.883424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:05.883424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:05.883424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:05.883424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:05.883424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:05.883424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:05.883424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:05.883424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:05.883424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:05.883424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:05.886819 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 316/622] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:46:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:05.887656 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd352ac-eefe-4595-99f7-4c2c85246b8c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:05.913066 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555ad69-b744-4fd0-9479-c8fcd3901300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:05.915130 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555ad69-b744-4fd0-9479-c8fcd3901300 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:05.915877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555ad69-b744-4fd0-9479-c8fcd3901300 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:05.916813 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555ad69-b744-4fd0-9479-c8fcd3901300 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:05.930187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555ad69-b744-4fd0-9479-c8fcd3901300 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:05.931895 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555ad69-b744-4fd0-9479-c8fcd3901300 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:05.949540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e555ad69-b744-4fd0-9479-c8fcd3901300 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:05.949540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:05.949540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:05.949540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:05.949540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:05.949540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:05.949540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:05.949540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:05.949540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:05.949540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:05.949540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:05.949540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:05.949540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:05.949540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:05.949540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:05.950534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:05.950534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:05.950534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:05.950534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:05.950534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:05.950534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:05.950534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:05.950534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:05.950534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:05.950534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:05.950534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:05.950534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:05.950534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:05.950534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:05.950534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:05.950534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:05.950534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:05.951855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:05.951855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:05.951855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:05.951855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:05.951855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 307/623] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:05.951855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555ad69-b744-4fd0-9479-c8fcd3901300 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:06.024555 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd352ac-eefe-4595-99f7-4c2c85246b8c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:06.051957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:06.080556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797aef1-35ba-4858-bc2c-6df4443f470d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:06.081365 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797aef1-35ba-4858-bc2c-6df4443f470d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:06.081649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797aef1-35ba-4858-bc2c-6df4443f470d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:06.081924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797aef1-35ba-4858-bc2c-6df4443f470d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:06.093795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797aef1-35ba-4858-bc2c-6df4443f470d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:06.095352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797aef1-35ba-4858-bc2c-6df4443f470d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:06.096532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555ad69-b744-4fd0-9479-c8fcd3901300 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:06.113242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-0797aef1-35ba-4858-bc2c-6df4443f470d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:06.113242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:06.113242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:06.113242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:06.113242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:06.113242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:06.113242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:06.113242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:06.113242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:06.113242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:06.113242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:06.113242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:06.113242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:06.113242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:06.113242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:06.114200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:06.114200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:06.114200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:06.114200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:06.114200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:06.114200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:06.114200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:06.114200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:06.114200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:06.114200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:06.114200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:06.114200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:06.114200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:06.114200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:06.114200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:06.114200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:06.114200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:06.115126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:06.115126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:06.115126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:06.115126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:06.115126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 317/624] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:06.115413 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797aef1-35ba-4858-bc2c-6df4443f470d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:06.121087 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:06.146073 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925915f-da71-47b4-b12e-1d9ab90e948e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:06.147722 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925915f-da71-47b4-b12e-1d9ab90e948e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:06.148585 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925915f-da71-47b4-b12e-1d9ab90e948e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:06.149101 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925915f-da71-47b4-b12e-1d9ab90e948e None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:06.150732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925915f-da71-47b4-b12e-1d9ab90e948e None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:06.151081 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925915f-da71-47b4-b12e-1d9ab90e948e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:06.153267 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925915f-da71-47b4-b12e-1d9ab90e948e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:06.163900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 308/625] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:46:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:06.164834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925915f-da71-47b4-b12e-1d9ab90e948e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:06.242928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797aef1-35ba-4858-bc2c-6df4443f470d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:06.284267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f73ee-02eb-402b-8ddf-086223995524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:06.285885 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f73ee-02eb-402b-8ddf-086223995524 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:06.286732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f73ee-02eb-402b-8ddf-086223995524 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:06.287217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f73ee-02eb-402b-8ddf-086223995524 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:06.288492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f73ee-02eb-402b-8ddf-086223995524 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:06.289539 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f73ee-02eb-402b-8ddf-086223995524 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:06.291796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f73ee-02eb-402b-8ddf-086223995524 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:06.298453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925915f-da71-47b4-b12e-1d9ab90e948e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:06.310179 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 318/626] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:06.311383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f73ee-02eb-402b-8ddf-086223995524 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:06.342021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456d5df-aff3-4474-889c-930e112cb411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:06.343302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456d5df-aff3-4474-889c-930e112cb411 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:06.343817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456d5df-aff3-4474-889c-930e112cb411 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:06.344546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456d5df-aff3-4474-889c-930e112cb411 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:06.345923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456d5df-aff3-4474-889c-930e112cb411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:06.346667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456d5df-aff3-4474-889c-930e112cb411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:06.348512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456d5df-aff3-4474-889c-930e112cb411 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:06.353045 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:06.373833 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 309/627] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:06.374912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456d5df-aff3-4474-889c-930e112cb411 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:06.422413 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:06.446649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f73ee-02eb-402b-8ddf-086223995524 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:06.485142 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf82ba2-988d-4f89-9833-ae9723ebef2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:06.486462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf82ba2-988d-4f89-9833-ae9723ebef2d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:06.486778 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf82ba2-988d-4f89-9833-ae9723ebef2d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:06.487056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf82ba2-988d-4f89-9833-ae9723ebef2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:06.498665 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf82ba2-988d-4f89-9833-ae9723ebef2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:06.500312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf82ba2-988d-4f89-9833-ae9723ebef2d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:06.503278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456d5df-aff3-4474-889c-930e112cb411 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:06.519158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-0cf82ba2-988d-4f89-9833-ae9723ebef2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:06.519158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:06.519158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:06.519158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:06.519158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:06.519158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:06.519158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:06.519158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:06.519158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:06.519158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:06.519158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:06.519158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:06.519158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:06.519158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:06.519158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:06.520109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:06.520109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:06.520109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:06.520109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:06.520109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:06.520109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:06.520109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:06.520109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:06.520109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:06.520109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:06.520109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:06.520109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:06.520109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:06.520109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:06.520109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:06.520109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:06.520109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:06.521080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:06.521080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:06.521080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:06.521080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:06.521080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 319/628] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:06.521369 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf82ba2-988d-4f89-9833-ae9723ebef2d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:06.541424 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb9ce1-e245-48d0-8943-3160df670016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:06.542214 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb9ce1-e245-48d0-8943-3160df670016 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:06.542484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb9ce1-e245-48d0-8943-3160df670016 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:06.542779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb9ce1-e245-48d0-8943-3160df670016 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:06.549498 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb9ce1-e245-48d0-8943-3160df670016 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:06.550963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb9ce1-e245-48d0-8943-3160df670016 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:06.565484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-abfb9ce1-e245-48d0-8943-3160df670016 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:06.565484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:06.565484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:06.565484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:06.565484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:06.565484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:06.565484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:06.565484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:06.565484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:06.565484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:06.565484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:06.565484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:06.565484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:06.565484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:06.565484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:06.566423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:06.566423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:06.566423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:06.566423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:06.566423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:06.566423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:06.566423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:06.566423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:06.566423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:06.566423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:06.566423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:06.566423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:06.566423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:06.566423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:06.566423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:06.566423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:06.566423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:06.567340 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:06.567340 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:06.567340 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:06.567340 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:06.567340 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:06.567340 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:06.567340 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:06.567340 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:06.571065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 310/629] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:06.573433 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3c54f-4b8f-4bdf-a242-8c6371e69afb None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:06.573727 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3c54f-4b8f-4bdf-a242-8c6371e69afb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:06.574009 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3c54f-4b8f-4bdf-a242-8c6371e69afb None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:06.576249 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 311/630] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:06.576878 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3c54f-4b8f-4bdf-a242-8c6371e69afb None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:06.619953 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:06.637944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf82ba2-988d-4f89-9833-ae9723ebef2d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:06.677237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ddcea-6dc4-4b84-96de-2835ffac8730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:06.678026 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ddcea-6dc4-4b84-96de-2835ffac8730 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:06.678303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ddcea-6dc4-4b84-96de-2835ffac8730 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:06.678602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ddcea-6dc4-4b84-96de-2835ffac8730 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:06.690697 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ddcea-6dc4-4b84-96de-2835ffac8730 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:06.692185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ddcea-6dc4-4b84-96de-2835ffac8730 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:06.696915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:06.701878 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3c54f-4b8f-4bdf-a242-8c6371e69afb None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:06.707176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-594ddcea-6dc4-4b84-96de-2835ffac8730 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:46:06.707176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:06.707176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:06.707176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:06.707176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:06.707176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:06.707176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:06.707176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:06.707176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:06.707176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:06.707176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:06.707176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:06.707176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:06.707176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:06.707176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:06.707176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:06.708279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:06.709252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:06.709252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:06.709252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:06.709252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:06.709252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:06.709252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:06.709252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:06.709252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:06.709252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:46:06.709252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:06.709252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 320/631] 10.222.0.70 () {56 vars in 1093 bytes} [Thu Jun 16 18:46:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:06.709864 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ddcea-6dc4-4b84-96de-2835ffac8730 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:06.739977 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3014ae-a62d-4feb-933f-f2ab09959cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:06.740641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3014ae-a62d-4feb-933f-f2ab09959cae None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:06.740934 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3014ae-a62d-4feb-933f-f2ab09959cae None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:06.741209 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3014ae-a62d-4feb-933f-f2ab09959cae None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:06.741959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3014ae-a62d-4feb-933f-f2ab09959cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:06.742260 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3014ae-a62d-4feb-933f-f2ab09959cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:06.743668 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3014ae-a62d-4feb-933f-f2ab09959cae None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:06.755692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 312/632] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:06.756829 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053756e-4595-4833-bd99-ae5b87e4df5a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:06.757148 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053756e-4595-4833-bd99-ae5b87e4df5a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:06.757422 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053756e-4595-4833-bd99-ae5b87e4df5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:06.773695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4053756e-4595-4833-bd99-ae5b87e4df5a None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:06.774911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4053756e-4595-4833-bd99-ae5b87e4df5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:06.813523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ddcea-6dc4-4b84-96de-2835ffac8730 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:06.841218 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 313/633] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:46:06.842683 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf4f05-b9a5-4188-9ce8-4809dd8f4a5b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:06.843002 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf4f05-b9a5-4188-9ce8-4809dd8f4a5b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:06.843283 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf4f05-b9a5-4188-9ce8-4809dd8f4a5b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:06.844543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 314/634] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:06.846684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf4f05-b9a5-4188-9ce8-4809dd8f4a5b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:06.851661 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32020bf4-a712-4dfb-8700-d13c2f6647a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:06.852325 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32020bf4-a712-4dfb-8700-d13c2f6647a4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:06.852609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32020bf4-a712-4dfb-8700-d13c2f6647a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:06.852888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32020bf4-a712-4dfb-8700-d13c2f6647a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:06.853688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32020bf4-a712-4dfb-8700-d13c2f6647a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:06.853989 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32020bf4-a712-4dfb-8700-d13c2f6647a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:06.855453 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32020bf4-a712-4dfb-8700-d13c2f6647a4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:06.868074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 321/635] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:06.868890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32020bf4-a712-4dfb-8700-d13c2f6647a4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:06.875850 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:06.940823 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:06.952014 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf4f05-b9a5-4188-9ce8-4809dd8f4a5b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:06.979738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32020bf4-a712-4dfb-8700-d13c2f6647a4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:06.987848 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52836ce-efe3-490f-ae68-82d52879ea7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:06.988513 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52836ce-efe3-490f-ae68-82d52879ea7f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:06.988791 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52836ce-efe3-490f-ae68-82d52879ea7f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:06.989062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52836ce-efe3-490f-ae68-82d52879ea7f None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:06.989890 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52836ce-efe3-490f-ae68-82d52879ea7f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:06.990194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52836ce-efe3-490f-ae68-82d52879ea7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:06.991670 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52836ce-efe3-490f-ae68-82d52879ea7f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:07.008994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 315/636] 10.222.0.70 () {56 vars in 1133 bytes} [Thu Jun 16 18:46:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:07.010118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf8011-5fb9-4fa1-847b-4c8d0e128495 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:07.010404 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf8011-5fb9-4fa1-847b-4c8d0e128495 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:07.010708 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf8011-5fb9-4fa1-847b-4c8d0e128495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:07.013618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eef781-4a48-494c-aeca-848fc71a773c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:07.014194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eef781-4a48-494c-aeca-848fc71a773c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:07.014477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eef781-4a48-494c-aeca-848fc71a773c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:07.014788 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eef781-4a48-494c-aeca-848fc71a773c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:07.025061 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eef781-4a48-494c-aeca-848fc71a773c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:07.026517 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eef781-4a48-494c-aeca-848fc71a773c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:07.027046 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-53cf8011-5fb9-4fa1-847b-4c8d0e128495 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:07.028213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-53cf8011-5fb9-4fa1-847b-4c8d0e128495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:07.041032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-43eef781-4a48-494c-aeca-848fc71a773c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:46:07.041032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:07.041032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:07.041032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:07.041032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:07.041032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:07.041032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:07.041032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:07.041032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:07.041032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:07.041032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:07.041032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:07.041032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:07.041032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:07.041032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:07.041032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:07.042168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:07.043389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:07.043389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:07.043389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:07.043389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:07.043389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:07.043389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:07.043389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:07.043389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:07.043389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:46:07.043389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:07.043389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 322/637] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:46:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:07.043389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eef781-4a48-494c-aeca-848fc71a773c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:07.100273 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 316/638] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:46:07.101339 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf8011-5fb9-4fa1-847b-4c8d0e128495 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:07.106559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:07.145999 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eef781-4a48-494c-aeca-848fc71a773c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:07.180896 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef42a66-0a8c-45bc-9432-0b5659752c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:07.181657 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:07.182016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef42a66-0a8c-45bc-9432-0b5659752c70 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:07.182322 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef42a66-0a8c-45bc-9432-0b5659752c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:07.182670 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef42a66-0a8c-45bc-9432-0b5659752c70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:07.193768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef42a66-0a8c-45bc-9432-0b5659752c70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:07.201900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-7ef42a66-0a8c-45bc-9432-0b5659752c70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:07.201900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:07.201900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:07.201900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:07.201900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:07.201900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:07.201900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:07.201900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:07.201900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:07.201900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:07.201900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:07.201900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:07.201900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:07.201900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:07.201900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:07.201900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:07.203029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:07.203029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:07.203029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:07.203029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:07.203029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:07.203029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:07.203029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:07.203029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:07.203029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:07.203029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:07.203029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:07.203029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:07.203029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:07.203029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:07.203029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:07.203029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:07.209119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:07.209119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:07.209119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:07.209119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:07.209119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:07.209119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:07.209119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:07.209119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:07.209119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:07.209119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:07.209119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:07.209119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:07.209119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 323/639] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:46:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:07.209967 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf8011-5fb9-4fa1-847b-4c8d0e128495 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:07.210072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef42a66-0a8c-45bc-9432-0b5659752c70 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:07.238079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc4a90-4ea9-47c3-9be1-54e0fc5d27fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:07.238991 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc4a90-4ea9-47c3-9be1-54e0fc5d27fd None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:07.239277 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc4a90-4ea9-47c3-9be1-54e0fc5d27fd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:07.239553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc4a90-4ea9-47c3-9be1-54e0fc5d27fd None admin] PATH_INFO: `/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/7b22fe0ab0354385924923d0d443acae/roles/c123034c11cc4d9ea6cab00e05bdcd53` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:07.254205 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc4a90-4ea9-47c3-9be1-54e0fc5d27fd None admin] RBAC: Authorizing `identity:create_grant(project_id=34185125e75d4aad8ad5382ab22ee8a2, user_id=7b22fe0ab0354385924923d0d443acae, role_id=c123034c11cc4d9ea6cab00e05bdcd53)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:07.255682 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc4a90-4ea9-47c3-9be1-54e0fc5d27fd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:07.269159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 317/640] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:46:07 2022] PUT /identity/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/7b22fe0ab0354385924923d0d443acae/roles/c123034c11cc4d9ea6cab00e05bdcd53 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:46:07.269693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc4a90-4ea9-47c3-9be1-54e0fc5d27fd None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:07.319134 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef42a66-0a8c-45bc-9432-0b5659752c70 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:07.353467 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:07.366420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6c970-e3a7-435c-8176-e9c276f902fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:07.367228 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6c970-e3a7-435c-8176-e9c276f902fa None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:07.367619 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6c970-e3a7-435c-8176-e9c276f902fa None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:07.368104 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6c970-e3a7-435c-8176-e9c276f902fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:07.385339 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc4a90-4ea9-47c3-9be1-54e0fc5d27fd None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:07.417777 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:07.428274 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d35b52-5b81-4ee6-b7b3-7d9c164fc02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:07.429983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d35b52-5b81-4ee6-b7b3-7d9c164fc02c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:07.430504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d35b52-5b81-4ee6-b7b3-7d9c164fc02c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:07.430989 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d35b52-5b81-4ee6-b7b3-7d9c164fc02c None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:07.432100 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d35b52-5b81-4ee6-b7b3-7d9c164fc02c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:07.432468 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d35b52-5b81-4ee6-b7b3-7d9c164fc02c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:07.434005 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d35b52-5b81-4ee6-b7b3-7d9c164fc02c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:07.451046 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6c970-e3a7-435c-8176-e9c276f902fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:07.454694 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6c970-e3a7-435c-8176-e9c276f902fa None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:07.457968 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 318/641] 10.222.0.70 () {56 vars in 1137 bytes} [Thu Jun 16 18:46:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:07.458730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d35b52-5b81-4ee6-b7b3-7d9c164fc02c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:07.570158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 324/642] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:46:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:46:07.570158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6c970-e3a7-435c-8176-e9c276f902fa None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:07.577204 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d35b52-5b81-4ee6-b7b3-7d9c164fc02c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:07.593343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:07.617857 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225dcc7-8f83-4345-a2a6-af7ca0ba9c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:07.618669 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225dcc7-8f83-4345-a2a6-af7ca0ba9c00 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:07.618957 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225dcc7-8f83-4345-a2a6-af7ca0ba9c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:07.619232 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225dcc7-8f83-4345-a2a6-af7ca0ba9c00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:07.630774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225dcc7-8f83-4345-a2a6-af7ca0ba9c00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:07.638628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-1225dcc7-8f83-4345-a2a6-af7ca0ba9c00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:07.638628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:07.638628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:07.638628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:07.638628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:07.638628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:07.638628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:07.638628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:07.638628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:07.638628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:07.638628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:07.638628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:07.638628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:07.638628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:07.638628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:07.638628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:07.639630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:07.639630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:07.639630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:07.639630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:07.639630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:07.639630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:07.639630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:07.639630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:07.639630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:07.639630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:07.639630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:07.639630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:07.639630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:07.639630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:07.639630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:07.639630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:07.640502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:07.640502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:07.640502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:07.640502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:07.640502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:07.640502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:07.640502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:07.640502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:07.640502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:07.640502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:07.640502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:07.640502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:07.640502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 319/643] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:46:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:07.641870 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225dcc7-8f83-4345-a2a6-af7ca0ba9c00 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:07.683152 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6c970-e3a7-435c-8176-e9c276f902fa None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:07.689916 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:07.721029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b18ae29-0089-4bcb-aa48-cc79883daabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:07.721700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18ae29-0089-4bcb-aa48-cc79883daabd None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:07.722103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18ae29-0089-4bcb-aa48-cc79883daabd None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:07.722381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18ae29-0089-4bcb-aa48-cc79883daabd None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:07.723156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b18ae29-0089-4bcb-aa48-cc79883daabd None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:07.723617 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b18ae29-0089-4bcb-aa48-cc79883daabd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:07.725070 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b18ae29-0089-4bcb-aa48-cc79883daabd None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:07.739057 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 325/644] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:46:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:07.739057 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b18ae29-0089-4bcb-aa48-cc79883daabd None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:07.751787 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225dcc7-8f83-4345-a2a6-af7ca0ba9c00 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:07.789195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940b62e-7f9a-4328-b99e-494e78771db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:07.789848 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940b62e-7f9a-4328-b99e-494e78771db4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:07.790131 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940b62e-7f9a-4328-b99e-494e78771db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:07.790405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940b62e-7f9a-4328-b99e-494e78771db4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:07.824975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:07.844215 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b18ae29-0089-4bcb-aa48-cc79883daabd None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:07.864618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940b62e-7f9a-4328-b99e-494e78771db4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:07.866167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940b62e-7f9a-4328-b99e-494e78771db4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:07.879852 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e68e5-8c78-4351-ae11-2be585631246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:07.880642 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e68e5-8c78-4351-ae11-2be585631246 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:07.880935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e68e5-8c78-4351-ae11-2be585631246 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:07.881198 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e68e5-8c78-4351-ae11-2be585631246 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:07.894201 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e68e5-8c78-4351-ae11-2be585631246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:07.895668 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e68e5-8c78-4351-ae11-2be585631246 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:07.915120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:07.916205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-868e68e5-8c78-4351-ae11-2be585631246 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:07.916205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:07.916205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:07.916205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:07.916205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:07.916205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:07.916205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:07.916205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:07.916205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:07.916205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:07.916205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:07.916205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:07.916205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:07.916205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:07.916205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:07.917343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:07.917343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:07.917343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:07.917343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:07.917343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:07.917343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:07.917343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:07.917343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:07.917343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:07.917343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:07.917343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:07.917343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:07.917343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:07.917343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:07.917343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:07.917343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:07.917343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:07.918875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:07.918875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:07.918875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:07.918875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:07.918875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 326/645] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:07.918875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e68e5-8c78-4351-ae11-2be585631246 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:07.986946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 320/646] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:46:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:46:07.987950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940b62e-7f9a-4328-b99e-494e78771db4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:08.043358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e68e5-8c78-4351-ae11-2be585631246 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:08.081358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bedfde-945e-43b0-b734-81e8d4677ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:08.082400 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bedfde-945e-43b0-b734-81e8d4677ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:08.082868 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bedfde-945e-43b0-b734-81e8d4677ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:08.083293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bedfde-945e-43b0-b734-81e8d4677ca4 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:08.084178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bedfde-945e-43b0-b734-81e8d4677ca4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:08.084745 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bedfde-945e-43b0-b734-81e8d4677ca4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:08.086553 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bedfde-945e-43b0-b734-81e8d4677ca4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:08.092556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:08.094266 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940b62e-7f9a-4328-b99e-494e78771db4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:08.097093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 327/647] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:46:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:08.098083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bedfde-945e-43b0-b734-81e8d4677ca4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:08.135244 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602e523-693a-4f4e-9dc2-2815470e7ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:08.136039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602e523-693a-4f4e-9dc2-2815470e7ffc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:08.136318 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602e523-693a-4f4e-9dc2-2815470e7ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:08.136591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602e523-693a-4f4e-9dc2-2815470e7ffc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:08.147057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602e523-693a-4f4e-9dc2-2815470e7ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:08.149129 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602e523-693a-4f4e-9dc2-2815470e7ffc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:08.155795 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:08.168714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-9602e523-693a-4f4e-9dc2-2815470e7ffc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:08.168714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:08.168714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:08.168714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:08.168714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:08.168714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:08.168714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:08.168714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:08.168714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:08.168714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:08.168714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:08.168714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:08.168714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:08.168714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:08.168714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:08.169840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:08.169840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:08.169840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:08.169840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:08.169840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:08.169840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:08.169840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:08.169840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:08.169840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:08.169840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:08.169840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:08.169840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:08.169840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:08.169840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:08.169840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:08.169840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:08.169840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:08.170994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:08.170994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:08.170994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:08.170994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:08.170994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 321/648] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:08.175209 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602e523-693a-4f4e-9dc2-2815470e7ffc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:08.208854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bedfde-945e-43b0-b734-81e8d4677ca4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:08.243847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c6e00-99a0-4d2c-b19f-3c8c34ea1217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:08.244722 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c6e00-99a0-4d2c-b19f-3c8c34ea1217 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:08.245180 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c6e00-99a0-4d2c-b19f-3c8c34ea1217 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:08.245611 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c6e00-99a0-4d2c-b19f-3c8c34ea1217 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:08.246332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c6e00-99a0-4d2c-b19f-3c8c34ea1217 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:08.246670 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c6e00-99a0-4d2c-b19f-3c8c34ea1217 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:08.248036 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c6e00-99a0-4d2c-b19f-3c8c34ea1217 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:08.260499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 328/649] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:08.261129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c6e00-99a0-4d2c-b19f-3c8c34ea1217 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:08.284054 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602e523-693a-4f4e-9dc2-2815470e7ffc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:08.320932 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e48a6f-2db4-453a-9dc7-f8a54af40aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:08.321690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e48a6f-2db4-453a-9dc7-f8a54af40aaa None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:08.321974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e48a6f-2db4-453a-9dc7-f8a54af40aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:08.322249 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e48a6f-2db4-453a-9dc7-f8a54af40aaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:08.323564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:08.333064 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e48a6f-2db4-453a-9dc7-f8a54af40aaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:08.334435 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e48a6f-2db4-453a-9dc7-f8a54af40aaa None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:08.354457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e9e48a6f-2db4-453a-9dc7-f8a54af40aaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:08.354457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:08.354457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:08.354457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:08.354457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:08.354457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:08.354457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:08.354457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:08.354457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:08.354457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:08.354457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:08.354457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:08.354457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:08.354457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:08.354457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:08.355434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:08.355434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:08.355434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:08.355434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:08.355434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:08.355434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:08.355434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:08.355434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:08.355434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:08.355434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:08.355434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:08.355434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:08.355434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:08.355434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:08.355434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:08.355434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:08.355434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:08.356334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:08.356334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:08.356334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:08.356334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:08.356334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 322/650] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:08.356617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e48a6f-2db4-453a-9dc7-f8a54af40aaa None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:08.376947 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c6e00-99a0-4d2c-b19f-3c8c34ea1217 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:08.415190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a19f6-c6c2-43d6-84f3-002a958b0c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:08.415716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:08.415982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a19f6-c6c2-43d6-84f3-002a958b0c01 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:08.416267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a19f6-c6c2-43d6-84f3-002a958b0c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:08.416543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a19f6-c6c2-43d6-84f3-002a958b0c01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:08.427424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a19f6-c6c2-43d6-84f3-002a958b0c01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:08.429378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a19f6-c6c2-43d6-84f3-002a958b0c01 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:08.448339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-6e3a19f6-c6c2-43d6-84f3-002a958b0c01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:08.448339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:08.448339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:08.448339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:08.448339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:08.448339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:08.448339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:08.448339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:08.448339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:08.448339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:08.448339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:08.448339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:08.448339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:08.448339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:08.448339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:08.449269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:08.449269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:08.449269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:08.449269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:08.449269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:08.449269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:08.449269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:08.449269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:08.449269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:08.449269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:08.449269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:08.449269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:08.449269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:08.449269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:08.449269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:08.449269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:08.449269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:08.450161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:08.450161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:08.450161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:08.450161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:08.454160 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 329/651] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:08.454956 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a19f6-c6c2-43d6-84f3-002a958b0c01 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:08.475939 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e48a6f-2db4-453a-9dc7-f8a54af40aaa None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:08.514876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258fe81-4218-458d-8ad2-d268ab0d8f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:08.515999 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258fe81-4218-458d-8ad2-d268ab0d8f94 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:08.516570 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258fe81-4218-458d-8ad2-d268ab0d8f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:08.516997 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258fe81-4218-458d-8ad2-d268ab0d8f94 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:08.518077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258fe81-4218-458d-8ad2-d268ab0d8f94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:08.518651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258fe81-4218-458d-8ad2-d268ab0d8f94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:08.519862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258fe81-4218-458d-8ad2-d268ab0d8f94 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:08.532261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 323/652] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:08.533222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258fe81-4218-458d-8ad2-d268ab0d8f94 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:08.573484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a19f6-c6c2-43d6-84f3-002a958b0c01 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:08.587082 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:08.610466 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e944e-3c65-40ce-9912-fbf1340919e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:08.611285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e944e-3c65-40ce-9912-fbf1340919e5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:08.611567 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e944e-3c65-40ce-9912-fbf1340919e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:08.611839 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e944e-3c65-40ce-9912-fbf1340919e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:08.618816 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e944e-3c65-40ce-9912-fbf1340919e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:08.620262 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e944e-3c65-40ce-9912-fbf1340919e5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:08.634897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ec7e944e-3c65-40ce-9912-fbf1340919e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:08.634897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:08.634897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:08.634897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:08.634897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:08.634897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:08.634897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:08.634897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:08.634897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:08.634897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:08.634897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:08.634897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:08.634897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:08.634897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:08.634897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:08.635895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:08.635895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:08.635895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:08.635895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:08.635895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:08.635895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:08.635895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:08.635895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:08.635895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:08.635895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:08.635895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:08.635895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:08.635895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:08.635895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:08.635895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:08.635895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:08.635895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:08.636797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:08.636797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:08.636797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:08.636797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:08.636797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:08.636797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:08.636797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:08.636797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:08.636797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 330/653] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:08.637281 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e944e-3c65-40ce-9912-fbf1340919e5 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:08.652482 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258fe81-4218-458d-8ad2-d268ab0d8f94 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:08.663039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:08.689701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ac9d0-43aa-4f08-97aa-ff202218b41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:08.690402 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ac9d0-43aa-4f08-97aa-ff202218b41f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:08.690741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ac9d0-43aa-4f08-97aa-ff202218b41f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:08.691043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ac9d0-43aa-4f08-97aa-ff202218b41f None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:08.691777 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ac9d0-43aa-4f08-97aa-ff202218b41f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:08.692082 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ac9d0-43aa-4f08-97aa-ff202218b41f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:08.693454 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ac9d0-43aa-4f08-97aa-ff202218b41f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:08.707280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 324/654] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:08.707904 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ac9d0-43aa-4f08-97aa-ff202218b41f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:08.763435 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e944e-3c65-40ce-9912-fbf1340919e5 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:08.800021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d70c6-d906-41bf-80ca-c13648194a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:08.801028 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d70c6-d906-41bf-80ca-c13648194a82 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:08.801369 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d70c6-d906-41bf-80ca-c13648194a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:08.801660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d70c6-d906-41bf-80ca-c13648194a82 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:08.802374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d70c6-d906-41bf-80ca-c13648194a82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:08.802704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d70c6-d906-41bf-80ca-c13648194a82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:08.804702 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d70c6-d906-41bf-80ca-c13648194a82 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:08.814566 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ac9d0-43aa-4f08-97aa-ff202218b41f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:08.817301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 331/655] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:08.818107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d70c6-d906-41bf-80ca-c13648194a82 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:08.852497 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:08.857740 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac031223-1e84-43f3-84ee-c11617cedd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:08.858764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac031223-1e84-43f3-84ee-c11617cedd3c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:08.859188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac031223-1e84-43f3-84ee-c11617cedd3c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:08.859604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac031223-1e84-43f3-84ee-c11617cedd3c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:08.867549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac031223-1e84-43f3-84ee-c11617cedd3c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:08.869565 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac031223-1e84-43f3-84ee-c11617cedd3c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:08.885021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ac031223-1e84-43f3-84ee-c11617cedd3c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:46:08.885021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:08.885021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:08.885021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:08.885021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:08.885021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:08.885021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:08.885021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:08.885021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:08.885021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:08.885021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:08.885021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:08.885021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:08.885021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:08.885021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:08.885021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:08.886193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:08.887385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:08.887385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:08.887385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:08.887385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:08.887385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:08.887385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:08.887385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:08.887385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:08.887385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:46:08.887385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:08.889676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 325/656] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:46:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:08.890698 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac031223-1e84-43f3-84ee-c11617cedd3c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:08.923110 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:08.931794 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d70c6-d906-41bf-80ca-c13648194a82 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:08.972067 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bc9ed-462c-4635-b8b4-bbafb925ec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:08.972736 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bc9ed-462c-4635-b8b4-bbafb925ec99 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:08.973021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bc9ed-462c-4635-b8b4-bbafb925ec99 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:08.973300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bc9ed-462c-4635-b8b4-bbafb925ec99 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:08.974112 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bc9ed-462c-4635-b8b4-bbafb925ec99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:08.974424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bc9ed-462c-4635-b8b4-bbafb925ec99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:08.976482 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bc9ed-462c-4635-b8b4-bbafb925ec99 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:08.992810 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 332/657] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:08.993828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bc9ed-462c-4635-b8b4-bbafb925ec99 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:09.008545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac031223-1e84-43f3-84ee-c11617cedd3c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:09.057026 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b7470-8330-4357-9b13-ab46e372c452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:09.057955 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b7470-8330-4357-9b13-ab46e372c452 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:09.058402 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b7470-8330-4357-9b13-ab46e372c452 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:09.058725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b7470-8330-4357-9b13-ab46e372c452 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:09.068027 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b7470-8330-4357-9b13-ab46e372c452 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:09.069539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b7470-8330-4357-9b13-ab46e372c452 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:09.086000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b18b7470-8330-4357-9b13-ab46e372c452 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:09.086000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:09.086000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:09.086000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:09.086000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:09.086000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:09.086000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:09.086000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:09.086000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:09.086000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:09.086000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:09.086000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:09.086000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:09.086000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:09.086000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:09.092395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:09.092395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:09.092395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:09.092395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:09.092395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:09.092395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:09.092395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:09.092395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:09.092395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:09.092395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:09.092395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:09.092395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:09.092395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:09.092395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:09.092395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:09.092395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:09.092395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:09.093330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:09.093330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:09.093330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:09.093330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:09.093330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:09.093330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:09.093330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:09.093330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:09.093330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 326/658] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:09.093330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b7470-8330-4357-9b13-ab46e372c452 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:09.094676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:09.099508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bc9ed-462c-4635-b8b4-bbafb925ec99 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:09.135232 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6120e-5889-4b87-a0c9-38c13d338014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:09.136053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6120e-5889-4b87-a0c9-38c13d338014 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:09.136402 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6120e-5889-4b87-a0c9-38c13d338014 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:09.136771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6120e-5889-4b87-a0c9-38c13d338014 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:09.144984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6120e-5889-4b87-a0c9-38c13d338014 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:09.146412 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6120e-5889-4b87-a0c9-38c13d338014 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:09.161700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-c5b6120e-5889-4b87-a0c9-38c13d338014 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:46:09.161700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:09.161700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:09.161700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:09.161700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:09.161700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:09.161700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:09.161700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:09.161700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:09.161700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:09.161700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:09.161700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:09.161700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:09.161700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:09.161700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:09.161700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:09.162768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:09.164167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:09.164167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:09.164167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:09.164167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:09.164167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:09.164167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:09.164167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:09.164167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:09.164167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:46:09.164167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:09.167577 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 333/659] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:46:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:09.168221 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6120e-5889-4b87-a0c9-38c13d338014 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:09.200411 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:09.212896 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b7470-8330-4357-9b13-ab46e372c452 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:09.249977 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acaf36-c236-4fed-a385-2409bb6f4fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:09.250688 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acaf36-c236-4fed-a385-2409bb6f4fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:09.250981 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acaf36-c236-4fed-a385-2409bb6f4fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:09.251260 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acaf36-c236-4fed-a385-2409bb6f4fe5 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:09.252097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acaf36-c236-4fed-a385-2409bb6f4fe5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:09.252406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acaf36-c236-4fed-a385-2409bb6f4fe5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:09.253951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acaf36-c236-4fed-a385-2409bb6f4fe5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:09.277123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6120e-5889-4b87-a0c9-38c13d338014 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:09.278370 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 327/660] 10.222.0.70 () {56 vars in 1139 bytes} [Thu Jun 16 18:46:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:09.279184 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acaf36-c236-4fed-a385-2409bb6f4fe5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:09.312587 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7b253-ada8-40c2-81fe-21bf662d3e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:09.313240 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7b253-ada8-40c2-81fe-21bf662d3e04 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:09.313524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7b253-ada8-40c2-81fe-21bf662d3e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:09.313801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7b253-ada8-40c2-81fe-21bf662d3e04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:09.314648 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7b253-ada8-40c2-81fe-21bf662d3e04 None admin] RBAC: Adding query filter params (role.id=c123034c11cc4d9ea6cab00e05bdcd53, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=8342da838fe4462b9de2817ad3ac4ff7) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:09.314963 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7b253-ada8-40c2-81fe-21bf662d3e04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:09.316385 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7b253-ada8-40c2-81fe-21bf662d3e04 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:09.323418 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 334/661] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:46:09 2022] GET /identity/v3/role_assignments?user.id=8342da838fe4462b9de2817ad3ac4ff7&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=c123034c11cc4d9ea6cab00e05bdcd53 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:09.324226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7b253-ada8-40c2-81fe-21bf662d3e04 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:09.357679 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:09.400361 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acaf36-c236-4fed-a385-2409bb6f4fe5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:09.442564 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4dc826-84dc-47dd-bb4d-6ad9f460d83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:09.443312 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dc826-84dc-47dd-bb4d-6ad9f460d83f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:09.443599 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dc826-84dc-47dd-bb4d-6ad9f460d83f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:09.443937 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dc826-84dc-47dd-bb4d-6ad9f460d83f None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:09.445368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4dc826-84dc-47dd-bb4d-6ad9f460d83f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:09.445684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4dc826-84dc-47dd-bb4d-6ad9f460d83f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:09.447189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4dc826-84dc-47dd-bb4d-6ad9f460d83f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:09.465120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 328/662] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:09.465120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4dc826-84dc-47dd-bb4d-6ad9f460d83f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:09.473530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:09.486189 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7b253-ada8-40c2-81fe-21bf662d3e04 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:09.529044 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4de662-0cf6-454b-b190-30adbe9afa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:09.529721 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4de662-0cf6-454b-b190-30adbe9afa01 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:09.530003 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4de662-0cf6-454b-b190-30adbe9afa01 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:09.530947 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4de662-0cf6-454b-b190-30adbe9afa01 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:09.531136 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4de662-0cf6-454b-b190-30adbe9afa01 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:09.531448 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4de662-0cf6-454b-b190-30adbe9afa01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:09.532969 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4de662-0cf6-454b-b190-30adbe9afa01 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:09.552483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 335/663] 10.222.0.70 () {56 vars in 1137 bytes} [Thu Jun 16 18:46:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:09.554857 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dd4f9-940d-4eec-bdf7-e77778934e34 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:09.555371 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dd4f9-940d-4eec-bdf7-e77778934e34 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:09.555660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dd4f9-940d-4eec-bdf7-e77778934e34 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:09.556935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 336/664] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:09.557638 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0dd4f9-940d-4eec-bdf7-e77778934e34 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:09.584011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4dc826-84dc-47dd-bb4d-6ad9f460d83f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:09.623155 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ea765-d5be-4eae-adf2-9463b2ce2d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:09.623811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea765-d5be-4eae-adf2-9463b2ce2d25 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:09.624094 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea765-d5be-4eae-adf2-9463b2ce2d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:09.624367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea765-d5be-4eae-adf2-9463b2ce2d25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:09.650532 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:09.681437 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0dd4f9-940d-4eec-bdf7-e77778934e34 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:09.698675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ea765-d5be-4eae-adf2-9463b2ce2d25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:09.700117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ea765-d5be-4eae-adf2-9463b2ce2d25 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:09.710881 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:09.719431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44055024-9013-4d95-8d90-8d1b6e9dab10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:09.720080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44055024-9013-4d95-8d90-8d1b6e9dab10 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:09.720356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44055024-9013-4d95-8d90-8d1b6e9dab10 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:09.720625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44055024-9013-4d95-8d90-8d1b6e9dab10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:09.721441 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44055024-9013-4d95-8d90-8d1b6e9dab10 None admin] RBAC: Adding query filter params (role.id=c123034c11cc4d9ea6cab00e05bdcd53, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=8563a25d16134d639a2a8849976fed9a) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:09.721750 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44055024-9013-4d95-8d90-8d1b6e9dab10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:09.723208 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44055024-9013-4d95-8d90-8d1b6e9dab10 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:09.738599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 337/665] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:46:09 2022] GET /identity/v3/role_assignments?user.id=8563a25d16134d639a2a8849976fed9a&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=c123034c11cc4d9ea6cab00e05bdcd53 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:09.738599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44055024-9013-4d95-8d90-8d1b6e9dab10 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:09.849382 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 329/666] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:46:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:46:09.849382 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf8324-e4ba-4813-95e3-e82e722c1a2f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:09.849382 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf8324-e4ba-4813-95e3-e82e722c1a2f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:09.849382 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf8324-e4ba-4813-95e3-e82e722c1a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:09.867180 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8ebf8324-e4ba-4813-95e3-e82e722c1a2f None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:09.868757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8ebf8324-e4ba-4813-95e3-e82e722c1a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:09.892096 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44055024-9013-4d95-8d90-8d1b6e9dab10 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:09.936688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8a5e9-30ce-4923-9db2-75421ce28f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:09.937341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8a5e9-30ce-4923-9db2-75421ce28f75 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:09.937707 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8a5e9-30ce-4923-9db2-75421ce28f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:09.937958 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8a5e9-30ce-4923-9db2-75421ce28f75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:09.949939 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 330/667] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:46:09.950868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf8324-e4ba-4813-95e3-e82e722c1a2f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:09.953901 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:09.997065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:10.035682 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8a5e9-30ce-4923-9db2-75421ce28f75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:10.037139 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8a5e9-30ce-4923-9db2-75421ce28f75 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:10.081253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf8324-e4ba-4813-95e3-e82e722c1a2f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:10.119061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e15c5-1c4d-4945-ba03-118e0cfae348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:10.120060 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e15c5-1c4d-4945-ba03-118e0cfae348 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:10.120252 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e15c5-1c4d-4945-ba03-118e0cfae348 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:10.120510 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e15c5-1c4d-4945-ba03-118e0cfae348 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:10.131377 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e15c5-1c4d-4945-ba03-118e0cfae348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:10.133869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e15c5-1c4d-4945-ba03-118e0cfae348 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:10.153098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c86e15c5-1c4d-4945-ba03-118e0cfae348 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:10.153098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:10.153098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:10.153098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:10.153098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:10.153098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:10.153098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:10.153098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:10.153098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:10.153098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:10.153098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:10.153098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:10.153098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:10.153098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:10.153098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:10.154068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:10.154068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:10.154068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:10.154068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:10.154068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:10.154068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:10.154068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:10.154068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:10.154068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:10.154068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:10.154068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:10.154068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:10.154068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:10.154068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:10.154068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:10.154068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:10.154068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:10.154993 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:10.154993 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:10.154993 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:10.154993 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:10.154993 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 331/668] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:10.155704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e15c5-1c4d-4945-ba03-118e0cfae348 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:10.162016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 338/669] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:46:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:46:10.163698 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8a5e9-30ce-4923-9db2-75421ce28f75 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:10.209402 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:10.263112 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:10.275429 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e15c5-1c4d-4945-ba03-118e0cfae348 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:10.279445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8a5e9-30ce-4923-9db2-75421ce28f75 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:10.313180 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a966aa2-f4c4-434d-aac3-2281dfa1049a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:10.314303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a966aa2-f4c4-434d-aac3-2281dfa1049a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:10.314792 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a966aa2-f4c4-434d-aac3-2281dfa1049a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:10.315226 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a966aa2-f4c4-434d-aac3-2281dfa1049a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:10.318153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ecb47e-37f5-434f-9321-a0ad2a319fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:10.318964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecb47e-37f5-434f-9321-a0ad2a319fab None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:10.319433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecb47e-37f5-434f-9321-a0ad2a319fab None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:10.319810 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecb47e-37f5-434f-9321-a0ad2a319fab None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:10.320656 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ecb47e-37f5-434f-9321-a0ad2a319fab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:10.321115 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ecb47e-37f5-434f-9321-a0ad2a319fab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:10.322532 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ecb47e-37f5-434f-9321-a0ad2a319fab None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:10.327426 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a966aa2-f4c4-434d-aac3-2281dfa1049a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:10.335500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-3a966aa2-f4c4-434d-aac3-2281dfa1049a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:10.335500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:10.335500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:10.335500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:10.335500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:10.335500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:10.335500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:10.335500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:10.335500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:10.335500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:10.335500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:10.335500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:10.335500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:10.335500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:10.335500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:10.335500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:10.336775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:10.336775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:10.336775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:10.336775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:10.336775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:10.336775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:10.336775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:10.336775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:10.336775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:10.336775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:10.336775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:10.336775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:10.336775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:10.336775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:10.336775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:10.336775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:10.337818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:10.337818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:10.337818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:10.337818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:10.337818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:10.337818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:10.337818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:10.337818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:10.337818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:10.337818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:10.337818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:10.337818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:10.339691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 339/670] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:10.340529 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 332/671] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:46:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:10.340705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ecb47e-37f5-434f-9321-a0ad2a319fab None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:10.351118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a966aa2-f4c4-434d-aac3-2281dfa1049a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:10.448891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:10.453335 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ecb47e-37f5-434f-9321-a0ad2a319fab None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:10.471813 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a966aa2-f4c4-434d-aac3-2281dfa1049a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:10.490864 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57401ee-283d-44b8-811b-c5e3823f174f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:10.492753 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57401ee-283d-44b8-811b-c5e3823f174f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:10.493257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57401ee-283d-44b8-811b-c5e3823f174f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:10.493725 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57401ee-283d-44b8-811b-c5e3823f174f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:10.505186 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57401ee-283d-44b8-811b-c5e3823f174f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:10.507512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57401ee-283d-44b8-811b-c5e3823f174f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:10.515259 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb863b5-cb24-435c-92e0-5b5b14fac8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:10.516068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb863b5-cb24-435c-92e0-5b5b14fac8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:10.516355 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb863b5-cb24-435c-92e0-5b5b14fac8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:10.516636 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb863b5-cb24-435c-92e0-5b5b14fac8b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:10.531785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb863b5-cb24-435c-92e0-5b5b14fac8b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:10.533665 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb863b5-cb24-435c-92e0-5b5b14fac8b5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:10.537695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-d57401ee-283d-44b8-811b-c5e3823f174f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:10.537695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:10.537695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:10.537695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:10.537695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:10.537695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:10.537695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:10.537695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:10.537695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:10.537695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:10.537695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:10.537695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:10.537695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:10.537695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:10.537695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:10.539579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:10.539579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:10.539579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:10.539579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:10.539579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:10.539579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:10.539579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:10.539579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:10.539579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:10.539579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:10.539579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:10.539579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:10.539579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:10.539579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:10.539579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:10.539579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:10.539579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:10.540507 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:10.540507 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:10.540507 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:10.540507 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:10.543314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 340/672] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:10.544416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57401ee-283d-44b8-811b-c5e3823f174f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:10.551571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-2bb863b5-cb24-435c-92e0-5b5b14fac8b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:10.551571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:10.551571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:10.551571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:10.551571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:10.551571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:10.551571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:10.551571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:10.551571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:10.551571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:10.551571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:10.551571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:10.551571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:10.551571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:10.551571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:10.552449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:10.552449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:10.552449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:10.552449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:10.552449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:10.552449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:10.552449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:10.552449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:10.552449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:10.552449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:10.552449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:10.552449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:10.552449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:10.552449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:10.552449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:10.552449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:10.552449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:10.553350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:10.553350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:10.553350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:10.553350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:10.553350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:10.553350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:10.553350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:10.553350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:10.557560 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 333/673] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:10.559013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb863b5-cb24-435c-92e0-5b5b14fac8b5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:10.574912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:10.663959 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57401ee-283d-44b8-811b-c5e3823f174f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:10.694290 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb863b5-cb24-435c-92e0-5b5b14fac8b5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:10.708592 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30504f62-3288-48a6-849b-c751e7e0ed46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:10.709634 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30504f62-3288-48a6-849b-c751e7e0ed46 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:10.710501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30504f62-3288-48a6-849b-c751e7e0ed46 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:10.710956 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30504f62-3288-48a6-849b-c751e7e0ed46 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:10.711845 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30504f62-3288-48a6-849b-c751e7e0ed46 None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:10.712158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30504f62-3288-48a6-849b-c751e7e0ed46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:10.714028 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30504f62-3288-48a6-849b-c751e7e0ed46 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:10.725256 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 341/674] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:46:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:10.726454 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30504f62-3288-48a6-849b-c751e7e0ed46 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:10.732215 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edd283-0ffc-4ed9-a825-9526847d105d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:10.732944 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edd283-0ffc-4ed9-a825-9526847d105d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:10.733233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edd283-0ffc-4ed9-a825-9526847d105d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:10.733578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edd283-0ffc-4ed9-a825-9526847d105d None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:10.734315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6edd283-0ffc-4ed9-a825-9526847d105d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:10.734722 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6edd283-0ffc-4ed9-a825-9526847d105d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:10.736250 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6edd283-0ffc-4ed9-a825-9526847d105d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:10.743502 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:10.754561 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 334/675] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:10.755492 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edd283-0ffc-4ed9-a825-9526847d105d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:10.834193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:10.859288 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30504f62-3288-48a6-849b-c751e7e0ed46 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:10.874093 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edd283-0ffc-4ed9-a825-9526847d105d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:10.901691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112e7a1-699a-4b92-b479-1cba72916585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:10.903374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112e7a1-699a-4b92-b479-1cba72916585 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:10.903815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112e7a1-699a-4b92-b479-1cba72916585 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:10.904526 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112e7a1-699a-4b92-b479-1cba72916585 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:10.905625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112e7a1-699a-4b92-b479-1cba72916585 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:10.906088 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112e7a1-699a-4b92-b479-1cba72916585 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:10.908283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112e7a1-699a-4b92-b479-1cba72916585 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:10.911576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605945d7-b411-47e3-a185-0702fab5f31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:10.912467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605945d7-b411-47e3-a185-0702fab5f31f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:10.912836 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605945d7-b411-47e3-a185-0702fab5f31f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:10.913745 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605945d7-b411-47e3-a185-0702fab5f31f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:10.928864 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 342/676] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:10.928864 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112e7a1-699a-4b92-b479-1cba72916585 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:10.931325 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605945d7-b411-47e3-a185-0702fab5f31f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:10.932770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605945d7-b411-47e3-a185-0702fab5f31f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:10.961842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-605945d7-b411-47e3-a185-0702fab5f31f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:10.961842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:10.961842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:10.961842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:10.961842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:10.961842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:10.961842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:10.961842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:10.961842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:10.961842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:10.961842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:10.961842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:10.961842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:10.961842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:10.961842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:10.962794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:10.962794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:10.962794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:10.962794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:10.962794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:10.962794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:10.962794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:10.962794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:10.962794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:10.962794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:10.962794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:10.962794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:10.962794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:10.962794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:10.962794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:10.962794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:10.962794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:10.963708 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:10.963708 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:10.963708 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:10.963708 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:10.967148 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 335/677] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:10.968684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605945d7-b411-47e3-a185-0702fab5f31f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:11.015234 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:11.054257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112e7a1-699a-4b92-b479-1cba72916585 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:11.090401 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605945d7-b411-47e3-a185-0702fab5f31f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:11.102562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228e9a4-662f-4e7e-bc43-d919a13a001b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:11.103730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228e9a4-662f-4e7e-bc43-d919a13a001b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:11.104287 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228e9a4-662f-4e7e-bc43-d919a13a001b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:11.104581 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228e9a4-662f-4e7e-bc43-d919a13a001b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:11.113530 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228e9a4-662f-4e7e-bc43-d919a13a001b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:11.115321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228e9a4-662f-4e7e-bc43-d919a13a001b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:11.122982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:11.131591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffddef98-6140-4f78-a01f-db19c747ccf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:11.132812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-a228e9a4-662f-4e7e-bc43-d919a13a001b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:11.132812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:11.132812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:11.132812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:11.132812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:11.132812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:11.132812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:11.132812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:11.132812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:11.132812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:11.132812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:11.132812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:11.132812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:11.132812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:11.132812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:11.133972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:11.133972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:11.133972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:11.133972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:11.133972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:11.133972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:11.133972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:11.133972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:11.133972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:11.133972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:11.133972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:11.133972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:11.133972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:11.133972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:11.133972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:11.133972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:11.133972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:11.134890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:11.134890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:11.134890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:11.134890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:11.134890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:11.134890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:11.134890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:11.134890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:11.135343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddef98-6140-4f78-a01f-db19c747ccf7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:11.135343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddef98-6140-4f78-a01f-db19c747ccf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:11.135343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddef98-6140-4f78-a01f-db19c747ccf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:11.135536 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 343/678] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:11.135536 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228e9a4-662f-4e7e-bc43-d919a13a001b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:11.139886 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffddef98-6140-4f78-a01f-db19c747ccf7 None admin] RBAC: Adding query filter params (role.id=c123034c11cc4d9ea6cab00e05bdcd53, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=02a83f6e7aa448d6984a78914c1748cf) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:11.140312 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffddef98-6140-4f78-a01f-db19c747ccf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:11.141843 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffddef98-6140-4f78-a01f-db19c747ccf7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:11.149570 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 336/679] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:46:10 2022] GET /identity/v3/role_assignments?user.id=02a83f6e7aa448d6984a78914c1748cf&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=c123034c11cc4d9ea6cab00e05bdcd53 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:11.150640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffddef98-6140-4f78-a01f-db19c747ccf7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:11.352329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228e9a4-662f-4e7e-bc43-d919a13a001b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:11.360832 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffddef98-6140-4f78-a01f-db19c747ccf7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:11.403291 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:11.414489 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454e346-e64e-40a6-9e1b-72b556af22a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:11.415205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454e346-e64e-40a6-9e1b-72b556af22a1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:11.415489 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454e346-e64e-40a6-9e1b-72b556af22a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:11.415766 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454e346-e64e-40a6-9e1b-72b556af22a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:11.416495 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454e346-e64e-40a6-9e1b-72b556af22a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:11.416805 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454e346-e64e-40a6-9e1b-72b556af22a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:11.418175 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454e346-e64e-40a6-9e1b-72b556af22a1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:11.421095 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485528c-f677-48b9-8b27-2d68e12eed8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:11.422023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485528c-f677-48b9-8b27-2d68e12eed8b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:11.422551 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485528c-f677-48b9-8b27-2d68e12eed8b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:11.423743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485528c-f677-48b9-8b27-2d68e12eed8b None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:11.424761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485528c-f677-48b9-8b27-2d68e12eed8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:11.425318 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485528c-f677-48b9-8b27-2d68e12eed8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:11.427009 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485528c-f677-48b9-8b27-2d68e12eed8b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:11.435982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 344/680] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:11.441895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454e346-e64e-40a6-9e1b-72b556af22a1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:11.450682 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 337/681] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:11.454181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485528c-f677-48b9-8b27-2d68e12eed8b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:11.523970 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:11.578967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454e346-e64e-40a6-9e1b-72b556af22a1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:11.590464 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485528c-f677-48b9-8b27-2d68e12eed8b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:11.618747 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32587577-8fb7-4d9c-8e12-286022029c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:11.619901 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32587577-8fb7-4d9c-8e12-286022029c19 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:11.620393 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32587577-8fb7-4d9c-8e12-286022029c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:11.620851 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32587577-8fb7-4d9c-8e12-286022029c19 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:11.629421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32587577-8fb7-4d9c-8e12-286022029c19 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:11.631384 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32587577-8fb7-4d9c-8e12-286022029c19 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:11.634967 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f74736-750a-4883-b584-f672fe011ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:11.635735 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f74736-750a-4883-b584-f672fe011ade None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:11.636137 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f74736-750a-4883-b584-f672fe011ade None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:11.637022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f74736-750a-4883-b584-f672fe011ade None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:11.637435 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f74736-750a-4883-b584-f672fe011ade None admin] RBAC: Adding query filter params (role.id=c123034c11cc4d9ea6cab00e05bdcd53, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=a47de73e3f3b4979bff83140bf287275) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:11.637821 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f74736-750a-4883-b584-f672fe011ade None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:11.639397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f74736-750a-4883-b584-f672fe011ade None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:11.650597 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 338/682] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:46:11 2022] GET /identity/v3/role_assignments?user.id=a47de73e3f3b4979bff83140bf287275&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=c123034c11cc4d9ea6cab00e05bdcd53 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:11.655874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-32587577-8fb7-4d9c-8e12-286022029c19 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:46:11.655874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:11.655874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:11.655874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:11.655874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:11.655874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:11.655874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:11.655874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:11.655874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:11.655874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:11.655874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:11.655874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:11.655874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:11.655874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:11.655874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:11.655874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:11.656902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:11.657861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:11.657861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:11.657861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:11.657861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:11.657861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:11.657861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:11.657861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:11.657861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:11.657861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:46:11.657861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:11.665738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 345/683] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:46:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:11.667271 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f74736-750a-4883-b584-f672fe011ade None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:11.721959 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc781a4e-c86e-4e99-a41d-d68cb0e347c1 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:11.722609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc781a4e-c86e-4e99-a41d-d68cb0e347c1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:11.723091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc781a4e-c86e-4e99-a41d-d68cb0e347c1 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:11.726938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 346/684] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:11.732002 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13301cd2-d422-440e-954e-5654a89f136e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:11.732614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13301cd2-d422-440e-954e-5654a89f136e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:11.733092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13301cd2-d422-440e-954e-5654a89f136e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:11.751295 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-13301cd2-d422-440e-954e-5654a89f136e None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:11.752859 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-13301cd2-d422-440e-954e-5654a89f136e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:11.807338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:11.849669 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f74736-750a-4883-b584-f672fe011ade None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:11.856769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 347/685] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:46:11.879191 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:11.911149 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181d696-cf98-4ff8-96e2-49ed5c779f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:11.912238 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181d696-cf98-4ff8-96e2-49ed5c779f20 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:11.912755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181d696-cf98-4ff8-96e2-49ed5c779f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:11.913992 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181d696-cf98-4ff8-96e2-49ed5c779f20 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:11.916042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13301cd2-d422-440e-954e-5654a89f136e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:11.928093 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181d696-cf98-4ff8-96e2-49ed5c779f20 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:11.929076 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181d696-cf98-4ff8-96e2-49ed5c779f20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:11.931639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181d696-cf98-4ff8-96e2-49ed5c779f20 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:11.957778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 339/686] 10.222.0.70 () {56 vars in 1135 bytes} [Thu Jun 16 18:46:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:11.958199 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181d696-cf98-4ff8-96e2-49ed5c779f20 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:12.059537 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13301cd2-d422-440e-954e-5654a89f136e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:12.101646 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3401a07-7dc7-4bf3-9323-0e1e736236e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:12.102460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3401a07-7dc7-4bf3-9323-0e1e736236e9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:12.102775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3401a07-7dc7-4bf3-9323-0e1e736236e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:12.103056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3401a07-7dc7-4bf3-9323-0e1e736236e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:12.111919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181d696-cf98-4ff8-96e2-49ed5c779f20 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:12.116600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3401a07-7dc7-4bf3-9323-0e1e736236e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:12.129798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-c3401a07-7dc7-4bf3-9323-0e1e736236e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:46:12.129798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:12.129798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:12.129798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:12.129798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:12.129798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:12.129798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:12.129798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:12.129798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:12.129798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:12.129798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:12.129798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:12.129798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:12.129798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:12.129798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:12.129798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:12.131073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:12.131073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:12.131073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:12.131073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:12.131073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:12.131073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:12.131073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:12.131073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:12.131073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:12.131073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:12.131073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:12.131073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:12.131073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:12.131073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:12.131073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:12.131073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:12.132004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:12.132004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:12.132004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:12.132004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:12.132004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:12.132004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:12.132004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:12.132004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:12.132004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:12.132004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:12.132004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:46:12.132004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:12.132004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 348/687] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:46:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:12.133051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614cf5f-16d5-491f-9a32-06add951a89a None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:12.133155 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614cf5f-16d5-491f-9a32-06add951a89a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:12.133464 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614cf5f-16d5-491f-9a32-06add951a89a None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:12.134993 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 349/688] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:12.146512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbeb63e-c975-41e1-a97b-97b68de5df23 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:12.146512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbeb63e-c975-41e1-a97b-97b68de5df23 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:12.146512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbeb63e-c975-41e1-a97b-97b68de5df23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:12.156598 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1069b9-9ff9-4a9c-896b-46a2d482128e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:12.157256 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1069b9-9ff9-4a9c-896b-46a2d482128e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:12.157546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1069b9-9ff9-4a9c-896b-46a2d482128e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:12.157821 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1069b9-9ff9-4a9c-896b-46a2d482128e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:12.162933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-bdbeb63e-c975-41e1-a97b-97b68de5df23 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:12.164359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-bdbeb63e-c975-41e1-a97b-97b68de5df23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:12.196628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:12.215687 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:12.241457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1069b9-9ff9-4a9c-896b-46a2d482128e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:12.243548 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1069b9-9ff9-4a9c-896b-46a2d482128e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:12.266645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 350/689] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:46:12.268240 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbeb63e-c975-41e1-a97b-97b68de5df23 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:12.353755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 340/690] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:46:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:46:12.354859 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1069b9-9ff9-4a9c-896b-46a2d482128e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:12.398867 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbeb63e-c975-41e1-a97b-97b68de5df23 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:12.430963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:12.435359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03695f-99ef-4153-a054-37519e3e1c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:12.436186 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03695f-99ef-4153-a054-37519e3e1c06 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:12.436594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03695f-99ef-4153-a054-37519e3e1c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:12.437035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03695f-99ef-4153-a054-37519e3e1c06 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:12.437905 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03695f-99ef-4153-a054-37519e3e1c06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:12.438352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03695f-99ef-4153-a054-37519e3e1c06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:12.439849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03695f-99ef-4153-a054-37519e3e1c06 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:12.453637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 351/691] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:46:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:12.454665 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03695f-99ef-4153-a054-37519e3e1c06 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:12.462129 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1069b9-9ff9-4a9c-896b-46a2d482128e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:12.490739 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:12.502011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841fd34-9b1f-4744-b195-56ce5c0d9863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:12.502898 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841fd34-9b1f-4744-b195-56ce5c0d9863 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:12.503182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841fd34-9b1f-4744-b195-56ce5c0d9863 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:12.503506 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841fd34-9b1f-4744-b195-56ce5c0d9863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:12.515549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841fd34-9b1f-4744-b195-56ce5c0d9863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:12.516994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841fd34-9b1f-4744-b195-56ce5c0d9863 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:12.537460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e841fd34-9b1f-4744-b195-56ce5c0d9863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:12.537460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:12.537460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:12.537460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:12.537460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:12.537460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:12.537460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:12.537460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:12.537460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:12.537460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:12.537460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:12.537460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:12.537460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:12.537460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:12.537460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:12.539434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:12.539434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:12.539434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:12.539434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:12.539434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:12.539434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:12.539434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:12.539434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:12.539434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:12.539434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:12.539434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:12.539434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:12.539434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:12.539434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:12.539434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:12.539434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:12.539434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:12.540381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:12.540381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:12.540381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:12.540381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:12.540381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 341/692] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:12.540381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841fd34-9b1f-4744-b195-56ce5c0d9863 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:12.572940 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03695f-99ef-4153-a054-37519e3e1c06 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:12.609113 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b2eff-0764-48d5-b2bd-fecd6e4730bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:12.609911 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b2eff-0764-48d5-b2bd-fecd6e4730bb None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:12.610215 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b2eff-0764-48d5-b2bd-fecd6e4730bb None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:12.610515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b2eff-0764-48d5-b2bd-fecd6e4730bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:12.622771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b2eff-0764-48d5-b2bd-fecd6e4730bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:12.624296 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b2eff-0764-48d5-b2bd-fecd6e4730bb None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:12.645124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-392b2eff-0764-48d5-b2bd-fecd6e4730bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:12.645124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:12.645124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:12.645124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:12.645124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:12.645124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:12.645124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:12.645124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:12.645124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:12.645124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:12.645124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:12.645124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:12.645124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:12.645124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:12.645124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:12.646206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:12.646206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:12.646206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:12.646206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:12.646206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:12.646206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:12.646206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:12.646206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:12.646206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:12.646206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:12.646206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:12.646206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:12.646206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:12.646206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:12.646206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:12.646206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:12.646206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:12.647150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:12.647150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:12.647150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:12.647150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:12.647150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 352/693] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:12.654820 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b2eff-0764-48d5-b2bd-fecd6e4730bb None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:12.655281 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841fd34-9b1f-4744-b195-56ce5c0d9863 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:12.677841 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:12.694284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841ee03-ebe3-4734-aaa2-b126e6b5c8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:12.695272 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841ee03-ebe3-4734-aaa2-b126e6b5c8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:12.695618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841ee03-ebe3-4734-aaa2-b126e6b5c8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:12.695953 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841ee03-ebe3-4734-aaa2-b126e6b5c8b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:12.707638 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841ee03-ebe3-4734-aaa2-b126e6b5c8b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:12.709271 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841ee03-ebe3-4734-aaa2-b126e6b5c8b3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:12.729950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-1841ee03-ebe3-4734-aaa2-b126e6b5c8b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:12.729950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:12.729950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:12.729950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:12.729950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:12.729950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:12.729950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:12.729950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:12.729950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:12.729950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:12.729950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:12.729950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:12.729950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:12.729950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:12.729950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:12.731759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:12.731759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:12.731759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:12.731759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:12.731759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:12.731759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:12.731759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:12.731759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:12.731759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:12.731759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:12.731759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:12.731759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:12.731759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:12.731759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:12.731759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:12.731759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:12.731759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:12.733757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:12.733757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:12.733757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:12.733757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:12.733757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 342/694] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:12.733757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841ee03-ebe3-4734-aaa2-b126e6b5c8b3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:12.755024 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:12.777433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b2eff-0764-48d5-b2bd-fecd6e4730bb None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:12.820119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9b104-7058-4d3b-ba68-09f11bce9cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:12.821270 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9b104-7058-4d3b-ba68-09f11bce9cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:12.821552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9b104-7058-4d3b-ba68-09f11bce9cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:12.822101 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9b104-7058-4d3b-ba68-09f11bce9cf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:12.822849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9b104-7058-4d3b-ba68-09f11bce9cf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:12.823159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9b104-7058-4d3b-ba68-09f11bce9cf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:12.825508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9b104-7058-4d3b-ba68-09f11bce9cf5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:12.840807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 353/695] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:12.841701 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9b104-7058-4d3b-ba68-09f11bce9cf5 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:12.870325 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841ee03-ebe3-4734-aaa2-b126e6b5c8b3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:12.908200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbccfda9-e5e1-429a-a334-aea304341039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:12.908842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccfda9-e5e1-429a-a334-aea304341039 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:12.909189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccfda9-e5e1-429a-a334-aea304341039 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:12.909559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccfda9-e5e1-429a-a334-aea304341039 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:12.910344 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbccfda9-e5e1-429a-a334-aea304341039 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:12.910677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbccfda9-e5e1-429a-a334-aea304341039 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:12.912096 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbccfda9-e5e1-429a-a334-aea304341039 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:12.924521 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 343/696] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:12.925319 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbccfda9-e5e1-429a-a334-aea304341039 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:12.970351 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:12.980388 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9b104-7058-4d3b-ba68-09f11bce9cf5 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:13.029999 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa409cde-27e8-4076-bf93-65b0a307e152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:13.030740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa409cde-27e8-4076-bf93-65b0a307e152 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:13.031050 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa409cde-27e8-4076-bf93-65b0a307e152 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:13.031332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa409cde-27e8-4076-bf93-65b0a307e152 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:13.032080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa409cde-27e8-4076-bf93-65b0a307e152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:13.032407 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa409cde-27e8-4076-bf93-65b0a307e152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:13.034364 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa409cde-27e8-4076-bf93-65b0a307e152 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:13.047238 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbccfda9-e5e1-429a-a334-aea304341039 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:13.055915 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 354/697] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:13.056849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa409cde-27e8-4076-bf93-65b0a307e152 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:13.067973 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:13.098629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb914fe-d82a-4f37-a993-36503dae1cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:13.103026 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb914fe-d82a-4f37-a993-36503dae1cb1 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:13.103329 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb914fe-d82a-4f37-a993-36503dae1cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:13.103588 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb914fe-d82a-4f37-a993-36503dae1cb1 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:13.104322 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb914fe-d82a-4f37-a993-36503dae1cb1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:13.105814 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb914fe-d82a-4f37-a993-36503dae1cb1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:13.138603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 344/698] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:46:12 2022] POST /identity/v3/users => generated 254 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:46:13.138603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb914fe-d82a-4f37-a993-36503dae1cb1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:13.241841 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa409cde-27e8-4076-bf93-65b0a307e152 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:13.294255 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:13.304795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda8134-46b3-4e3b-908d-889e41e1f32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:13.306233 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda8134-46b3-4e3b-908d-889e41e1f32f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:13.306861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda8134-46b3-4e3b-908d-889e41e1f32f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:13.307362 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda8134-46b3-4e3b-908d-889e41e1f32f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:13.316401 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda8134-46b3-4e3b-908d-889e41e1f32f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:13.318349 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda8134-46b3-4e3b-908d-889e41e1f32f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:13.331318 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb914fe-d82a-4f37-a993-36503dae1cb1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:13.343416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-cbda8134-46b3-4e3b-908d-889e41e1f32f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:13.343416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:13.343416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:13.343416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:13.343416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:13.343416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:13.343416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:13.343416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:13.343416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:13.343416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:13.343416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:13.343416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:13.343416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:13.343416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:13.343416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:13.344354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:13.344354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:13.344354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:13.344354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:13.344354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:13.344354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:13.344354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:13.344354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:13.344354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:13.344354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:13.344354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:13.344354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:13.344354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:13.344354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:13.344354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:13.344354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:13.344354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:13.345255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:13.345255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:13.345255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:13.345255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:13.345255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:13.345255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:13.345255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:13.345255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:13.346259 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 355/699] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:13.354625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda8134-46b3-4e3b-908d-889e41e1f32f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:13.385726 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60f300-fc25-48df-b826-baf08a58304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:13.386539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60f300-fc25-48df-b826-baf08a58304f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:13.386883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60f300-fc25-48df-b826-baf08a58304f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:13.387167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60f300-fc25-48df-b826-baf08a58304f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:13.396727 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60f300-fc25-48df-b826-baf08a58304f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:13.398350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60f300-fc25-48df-b826-baf08a58304f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:13.413705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-8c60f300-fc25-48df-b826-baf08a58304f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:46:13.413705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:13.413705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:13.413705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:13.413705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:13.413705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:13.413705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:13.413705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:13.413705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:13.413705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:13.413705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:13.413705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:13.413705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:13.413705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:13.413705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:13.413705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:13.414703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:13.416211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:13.416211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:13.416211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:13.416211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:13.416211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:13.416211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:13.416211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:13.416211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:13.416211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:46:13.416211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:13.416211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 345/700] 10.222.0.70 () {56 vars in 1093 bytes} [Thu Jun 16 18:46:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:13.422251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60f300-fc25-48df-b826-baf08a58304f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:13.424951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:13.463699 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda8134-46b3-4e3b-908d-889e41e1f32f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:13.501366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ebc7c-b5f2-41b8-8635-1bd9db665644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:13.501965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ebc7c-b5f2-41b8-8635-1bd9db665644 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:13.502250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ebc7c-b5f2-41b8-8635-1bd9db665644 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:13.502529 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ebc7c-b5f2-41b8-8635-1bd9db665644 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:13.503350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ebc7c-b5f2-41b8-8635-1bd9db665644 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:13.503713 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ebc7c-b5f2-41b8-8635-1bd9db665644 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:13.505154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ebc7c-b5f2-41b8-8635-1bd9db665644 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:13.520861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 356/701] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:13.528784 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ebc7c-b5f2-41b8-8635-1bd9db665644 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:13.545948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60f300-fc25-48df-b826-baf08a58304f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:13.593652 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d478083-aa96-4c43-8eba-548e726f6ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:13.594347 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d478083-aa96-4c43-8eba-548e726f6ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:13.594657 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d478083-aa96-4c43-8eba-548e726f6ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:13.594935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d478083-aa96-4c43-8eba-548e726f6ff0 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:13.595786 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d478083-aa96-4c43-8eba-548e726f6ff0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:13.596096 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d478083-aa96-4c43-8eba-548e726f6ff0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:13.597964 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d478083-aa96-4c43-8eba-548e726f6ff0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:13.622477 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 346/702] 10.222.0.70 () {56 vars in 1133 bytes} [Thu Jun 16 18:46:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:13.631738 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d478083-aa96-4c43-8eba-548e726f6ff0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:13.641834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:13.650779 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ebc7c-b5f2-41b8-8635-1bd9db665644 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:13.679444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:13.687739 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63ede9-b445-40ef-bdea-9efe32d339dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:13.688392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63ede9-b445-40ef-bdea-9efe32d339dc None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:13.688679 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63ede9-b445-40ef-bdea-9efe32d339dc None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:13.688960 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63ede9-b445-40ef-bdea-9efe32d339dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:13.689789 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63ede9-b445-40ef-bdea-9efe32d339dc None admin] RBAC: Adding query filter params (role.id=c123034c11cc4d9ea6cab00e05bdcd53, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=7b22fe0ab0354385924923d0d443acae) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:13.690111 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63ede9-b445-40ef-bdea-9efe32d339dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:13.691584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63ede9-b445-40ef-bdea-9efe32d339dc None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:13.714167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 357/703] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:46:13 2022] GET /identity/v3/role_assignments?user.id=7b22fe0ab0354385924923d0d443acae&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=c123034c11cc4d9ea6cab00e05bdcd53 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:13.817042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ce5cd-7a10-4fbf-abff-d90daae99774 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:13.817672 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ce5cd-7a10-4fbf-abff-d90daae99774 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:13.818176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ce5cd-7a10-4fbf-abff-d90daae99774 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:13.822223 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 358/704] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:46:13.826686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d478083-aa96-4c43-8eba-548e726f6ff0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:13.827244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8c89f-6bc9-41be-ad8a-50b69fbc7edd None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:13.827761 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8c89f-6bc9-41be-ad8a-50b69fbc7edd None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:13.828239 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8c89f-6bc9-41be-ad8a-50b69fbc7edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:13.847675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-67d8c89f-6bc9-41be-ad8a-50b69fbc7edd None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:13.849323 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-67d8c89f-6bc9-41be-ad8a-50b69fbc7edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:13.899116 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94038d82-19e6-4f8e-875f-ad8955ffe9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:13.900208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94038d82-19e6-4f8e-875f-ad8955ffe9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:13.900730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94038d82-19e6-4f8e-875f-ad8955ffe9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:13.902497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94038d82-19e6-4f8e-875f-ad8955ffe9b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:13.986612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 359/705] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:46:14.011302 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8c89f-6bc9-41be-ad8a-50b69fbc7edd None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:14.024951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94038d82-19e6-4f8e-875f-ad8955ffe9b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:14.026414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94038d82-19e6-4f8e-875f-ad8955ffe9b3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:14.036034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:14.122316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8c89f-6bc9-41be-ad8a-50b69fbc7edd None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:14.144479 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:14.149046 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 347/706] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:46:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:46:14.154887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94038d82-19e6-4f8e-875f-ad8955ffe9b3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:14.166727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72fb0b-4007-48c5-aa4d-40999fda13e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:14.168002 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72fb0b-4007-48c5-aa4d-40999fda13e5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:14.168651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72fb0b-4007-48c5-aa4d-40999fda13e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:14.169010 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72fb0b-4007-48c5-aa4d-40999fda13e5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:14.170054 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72fb0b-4007-48c5-aa4d-40999fda13e5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:14.171712 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72fb0b-4007-48c5-aa4d-40999fda13e5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:14.189340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-0d72fb0b-4007-48c5-aa4d-40999fda13e5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:46:14.189340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:14.189340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:14.189340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:14.189340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:14.189340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:14.189340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:14.189340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:14.189340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:46:14.189340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:14.189340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:14.189340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:14.189340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:14.189340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:14.189340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:14.189340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:14.189340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:14.190476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:14.190476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:14.190476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:14.190476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:14.190476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:14.190476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:14.190476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:14.190476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:14.190476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:14.190476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:14.190476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:14.190476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:14.190476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:14.190476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:14.190476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:14.190476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:14.191485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:14.191485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:14.191485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:14.191485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:14.191485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:14.191485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:14.191485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:14.191485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:14.191485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:14.191485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:14.191485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:14.191485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:14.191485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:46:14.191485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:14.193589 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 360/707] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:14 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:14.200530 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72fb0b-4007-48c5-aa4d-40999fda13e5 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:14.307810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94038d82-19e6-4f8e-875f-ad8955ffe9b3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:14.316194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:14.335766 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72fb0b-4007-48c5-aa4d-40999fda13e5 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:14.346622 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355bd57d-3ab4-4ecf-a36d-c49bda2f5d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:14.347569 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bd57d-3ab4-4ecf-a36d-c49bda2f5d53 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:14.347848 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bd57d-3ab4-4ecf-a36d-c49bda2f5d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:14.348119 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bd57d-3ab4-4ecf-a36d-c49bda2f5d53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:14.359816 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355bd57d-3ab4-4ecf-a36d-c49bda2f5d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:14.361383 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355bd57d-3ab4-4ecf-a36d-c49bda2f5d53 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:14.377606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad961922-7e74-43f1-bab4-40940a4be836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:14.378200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad961922-7e74-43f1-bab4-40940a4be836 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:14.378488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad961922-7e74-43f1-bab4-40940a4be836 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:14.378791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad961922-7e74-43f1-bab4-40940a4be836 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:14.379496 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad961922-7e74-43f1-bab4-40940a4be836 None admin] RBAC: Adding query filter params (name=network) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:14.379812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad961922-7e74-43f1-bab4-40940a4be836 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:14.380804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-355bd57d-3ab4-4ecf-a36d-c49bda2f5d53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:14.380804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:14.380804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:14.380804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:14.380804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:14.380804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:14.380804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:14.380804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:14.380804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:14.380804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:14.380804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:14.380804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:14.380804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:14.380804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:14.380804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:14.381889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:14.381889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:14.381889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:14.381889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:14.381889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:14.381889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:14.381889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:14.381889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:14.381889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:14.381889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:14.381889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:14.381889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:14.381889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:14.381889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:14.381889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:14.381889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:14.381889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:14.383251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:14.383251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:14.383251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:14.383251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:14.383484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad961922-7e74-43f1-bab4-40940a4be836 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:14.383558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 348/708] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:14.384707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bfdfa-451b-4de7-8611-21865b4ea26d None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:14.384707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bfdfa-451b-4de7-8611-21865b4ea26d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:14.384707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bfdfa-451b-4de7-8611-21865b4ea26d None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:14.386813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 361/710] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Jun 16 18:46:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:14.389825 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 349/710] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:14.391348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cea35-861c-4186-b20e-1801f25937de None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:14.391640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cea35-861c-4186-b20e-1801f25937de None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:14.391919 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cea35-861c-4186-b20e-1801f25937de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:14.395287 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0bfdfa-451b-4de7-8611-21865b4ea26d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:14.412248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5a9cea35-861c-4186-b20e-1801f25937de None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:14.414110 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5a9cea35-861c-4186-b20e-1801f25937de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:14.502883 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:14.507133 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 362/711] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:46:14.508395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cea35-861c-4186-b20e-1801f25937de None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:14.514634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0bfdfa-451b-4de7-8611-21865b4ea26d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:14.565891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa3b9c-c5c8-4750-8827-3f9e6af33467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:14.566559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa3b9c-c5c8-4750-8827-3f9e6af33467 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:14.566867 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa3b9c-c5c8-4750-8827-3f9e6af33467 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:14.567140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa3b9c-c5c8-4750-8827-3f9e6af33467 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:14.567864 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa3b9c-c5c8-4750-8827-3f9e6af33467 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:14.568176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa3b9c-c5c8-4750-8827-3f9e6af33467 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:14.570895 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa3b9c-c5c8-4750-8827-3f9e6af33467 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:14.596579 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 350/712] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:14.596579 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa3b9c-c5c8-4750-8827-3f9e6af33467 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:14.622363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:14.644166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cea35-861c-4186-b20e-1801f25937de None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:14.690093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6efd374-5082-4dd1-9c6f-e545de669308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:14.690954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efd374-5082-4dd1-9c6f-e545de669308 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:14.691243 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efd374-5082-4dd1-9c6f-e545de669308 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:14.691518 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efd374-5082-4dd1-9c6f-e545de669308 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:14.692230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6efd374-5082-4dd1-9c6f-e545de669308 None admin] RBAC: Adding query filter params (type=network) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:14.692898 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6efd374-5082-4dd1-9c6f-e545de669308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:14.695343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6efd374-5082-4dd1-9c6f-e545de669308 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:14.702774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 363/713] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Jun 16 18:46:14 2022] GET /identity/v3/services?type=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:14.703966 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6efd374-5082-4dd1-9c6f-e545de669308 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:14.725995 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa3b9c-c5c8-4750-8827-3f9e6af33467 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:14.762159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:14.805484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f634612-33c2-4ab4-be10-34a381715a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:14.806888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f634612-33c2-4ab4-be10-34a381715a22 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:14.807380 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f634612-33c2-4ab4-be10-34a381715a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:14.807879 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f634612-33c2-4ab4-be10-34a381715a22 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:14.808743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f634612-33c2-4ab4-be10-34a381715a22 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:14.810566 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f634612-33c2-4ab4-be10-34a381715a22 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:14.831742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6efd374-5082-4dd1-9c6f-e545de669308 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:14.840363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-5f634612-33c2-4ab4-be10-34a381715a22 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:46:14.840363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:14.840363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:14.840363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:14.840363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:14.840363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:14.840363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:14.840363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:14.840363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:46:14.840363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:14.840363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:14.840363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:14.840363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:14.840363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:14.840363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:14.840363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:14.841500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:14.841500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:14.841500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:14.841500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:14.841500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:14.841500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:14.841500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:14.841500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:14.841500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:14.841500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:14.841500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:14.841500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:14.841500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:14.841500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:14.841500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:14.841500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:14.842345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:14.842345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:14.842345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:14.842345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:14.842345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:14.842345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:14.842345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:14.842345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:14.842345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:14.842345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:14.842345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:14.842345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:14.842345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:14.842345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:46:14.842345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:14.845822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 351/714] 10.222.0.70 () {56 vars in 1129 bytes} [Thu Jun 16 18:46:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:14.855874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f634612-33c2-4ab4-be10-34a381715a22 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:14.886407 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bee7d-4d99-4828-a2f9-27233b693193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:14.887320 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bee7d-4d99-4828-a2f9-27233b693193 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:14.887614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bee7d-4d99-4828-a2f9-27233b693193 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:14.887888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bee7d-4d99-4828-a2f9-27233b693193 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:14.899637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6bee7d-4d99-4828-a2f9-27233b693193 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:14.901060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6bee7d-4d99-4828-a2f9-27233b693193 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:14.920608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-0a6bee7d-4d99-4828-a2f9-27233b693193 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:14.920608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:14.920608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:14.920608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:14.920608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:14.920608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:14.920608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:14.920608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:14.920608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:14.920608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:14.920608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:14.920608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:14.920608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:14.920608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:14.920608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:14.921627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:14.921627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:14.921627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:14.921627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:14.921627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:14.921627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:14.921627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:14.921627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:14.921627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:14.921627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:14.921627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:14.921627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:14.921627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:14.921627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:14.921627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:14.921627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:14.921627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:14.922566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:14.922566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:14.922566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:14.922566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:14.922566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:14.922566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:14.922566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:14.922566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:14.922566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 364/715] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:14.931839 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bee7d-4d99-4828-a2f9-27233b693193 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:15.011928 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:15.039752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f634612-33c2-4ab4-be10-34a381715a22 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:15.072240 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bee7d-4d99-4828-a2f9-27233b693193 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:15.073807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:15.092704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89455db-f6e2-45fa-a723-25787384e8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:15.093704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89455db-f6e2-45fa-a723-25787384e8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:15.094242 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89455db-f6e2-45fa-a723-25787384e8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:15.094763 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89455db-f6e2-45fa-a723-25787384e8f8 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:15.095682 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89455db-f6e2-45fa-a723-25787384e8f8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:15.096192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89455db-f6e2-45fa-a723-25787384e8f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:15.097768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89455db-f6e2-45fa-a723-25787384e8f8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:15.120734 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 352/716] 10.222.0.70 () {56 vars in 1124 bytes} [Thu Jun 16 18:46:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:15.126370 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89455db-f6e2-45fa-a723-25787384e8f8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:15.132742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d188c08-3b38-4969-98ee-49784cd5d63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:15.134678 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d188c08-3b38-4969-98ee-49784cd5d63b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:15.137171 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d188c08-3b38-4969-98ee-49784cd5d63b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:15.137687 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d188c08-3b38-4969-98ee-49784cd5d63b None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:15.138742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d188c08-3b38-4969-98ee-49784cd5d63b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:15.143386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d188c08-3b38-4969-98ee-49784cd5d63b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:15.145091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d188c08-3b38-4969-98ee-49784cd5d63b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:15.169209 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 365/717] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:15.175964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d188c08-3b38-4969-98ee-49784cd5d63b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:15.283102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89455db-f6e2-45fa-a723-25787384e8f8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:15.331498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d188c08-3b38-4969-98ee-49784cd5d63b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:15.332511 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1a7f3-e4be-4497-ae64-7570b89049d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:15.333419 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1a7f3-e4be-4497-ae64-7570b89049d3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:15.333918 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1a7f3-e4be-4497-ae64-7570b89049d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:15.334394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1a7f3-e4be-4497-ae64-7570b89049d3 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:15.336137 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1a7f3-e4be-4497-ae64-7570b89049d3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:15.336655 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1a7f3-e4be-4497-ae64-7570b89049d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:15.338256 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1a7f3-e4be-4497-ae64-7570b89049d3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:15.354502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 353/718] 10.222.0.70 () {56 vars in 1124 bytes} [Thu Jun 16 18:46:15 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:15.387495 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324a776-6506-4b02-ad19-82a2aa71d5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:15.388194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324a776-6506-4b02-ad19-82a2aa71d5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:15.388481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324a776-6506-4b02-ad19-82a2aa71d5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:15.388759 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324a776-6506-4b02-ad19-82a2aa71d5e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:15.389614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324a776-6506-4b02-ad19-82a2aa71d5e5 None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=8342da838fe4462b9de2817ad3ac4ff7) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:15.389931 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324a776-6506-4b02-ad19-82a2aa71d5e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:15.391444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324a776-6506-4b02-ad19-82a2aa71d5e5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:15.399821 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 366/719] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:46:15 2022] GET /identity/v3/role_assignments?user.id=8342da838fe4462b9de2817ad3ac4ff7&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:16.007788 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330aa50-2c4f-44b7-9081-8826f8ce3e0f None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:16.008438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330aa50-2c4f-44b7-9081-8826f8ce3e0f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:16.008943 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330aa50-2c4f-44b7-9081-8826f8ce3e0f None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:16.010823 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 354/720] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:16.015967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66937b90-79fd-46ca-8b6c-3e85a0c91626 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:16.016282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66937b90-79fd-46ca-8b6c-3e85a0c91626 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:16.016540 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66937b90-79fd-46ca-8b6c-3e85a0c91626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:16.043638 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-66937b90-79fd-46ca-8b6c-3e85a0c91626 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:16.044878 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-66937b90-79fd-46ca-8b6c-3e85a0c91626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:16.072143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:16.134049 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 367/721] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:46:16.163462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330aa50-2c4f-44b7-9081-8826f8ce3e0f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:16.182300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:16.292781 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330aa50-2c4f-44b7-9081-8826f8ce3e0f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:16.341965 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466e508-7640-40bb-bc5c-e3b54979403e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:16.342812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466e508-7640-40bb-bc5c-e3b54979403e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:16.343095 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466e508-7640-40bb-bc5c-e3b54979403e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:16.343367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466e508-7640-40bb-bc5c-e3b54979403e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:16.362081 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466e508-7640-40bb-bc5c-e3b54979403e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:16.370362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e466e508-7640-40bb-bc5c-e3b54979403e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:16.370362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:16.370362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:16.370362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:16.370362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:16.370362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:16.370362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:16.370362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:16.370362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:16.370362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:16.370362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:16.370362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:16.370362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:16.370362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:16.370362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:16.370362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:16.371389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:16.371389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:16.371389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:16.371389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:16.371389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:16.371389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:16.371389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:16.371389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:16.371389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:16.371389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:16.371389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:16.371389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:16.371389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:16.371389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:16.371389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:16.371389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:16.372228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:16.372228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:16.372228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:16.372228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:16.372228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:16.372228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:16.372228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:16.372228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:16.372228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:16.372228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:16.372228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:16.372228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:16.372228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 355/722] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:46:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:16.377880 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66937b90-79fd-46ca-8b6c-3e85a0c91626 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:16.497888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3125c-dac7-4902-9098-4716069b7045 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:16.498212 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3125c-dac7-4902-9098-4716069b7045 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:16.498446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3125c-dac7-4902-9098-4716069b7045 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:16.499843 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 356/723] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:46:16.504916 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6b903-44c9-4960-a180-2e74cf7df6ea None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:16.505200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6b903-44c9-4960-a180-2e74cf7df6ea None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:16.505477 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6b903-44c9-4960-a180-2e74cf7df6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:16.527484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66937b90-79fd-46ca-8b6c-3e85a0c91626 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:16.530009 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6bc6b903-44c9-4960-a180-2e74cf7df6ea None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:16.531089 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6bc6b903-44c9-4960-a180-2e74cf7df6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:16.590807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c25150-6bd7-4762-a0fb-efb7a01d5bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:16.591390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c25150-6bd7-4762-a0fb-efb7a01d5bab None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:16.592305 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c25150-6bd7-4762-a0fb-efb7a01d5bab None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:16.592592 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c25150-6bd7-4762-a0fb-efb7a01d5bab None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:16.593354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c25150-6bd7-4762-a0fb-efb7a01d5bab None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:16.593662 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c25150-6bd7-4762-a0fb-efb7a01d5bab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:16.595083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c25150-6bd7-4762-a0fb-efb7a01d5bab None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:16.608702 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 368/724] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:46:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:16.614067 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c25150-6bd7-4762-a0fb-efb7a01d5bab None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:16.645037 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:16.659175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:16.667940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 357/725] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:46:16.702199 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6b903-44c9-4960-a180-2e74cf7df6ea None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:16.775298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c25150-6bd7-4762-a0fb-efb7a01d5bab None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:16.822797 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6b903-44c9-4960-a180-2e74cf7df6ea None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:16.845068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb353e95-ecc5-4c67-8b35-caeaafa0681c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:16.846180 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb353e95-ecc5-4c67-8b35-caeaafa0681c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:16.846713 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb353e95-ecc5-4c67-8b35-caeaafa0681c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:16.847189 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb353e95-ecc5-4c67-8b35-caeaafa0681c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:16.869542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb353e95-ecc5-4c67-8b35-caeaafa0681c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:16.871335 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb353e95-ecc5-4c67-8b35-caeaafa0681c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:16.899109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-cb353e95-ecc5-4c67-8b35-caeaafa0681c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:16.899109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:16.899109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:16.899109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:16.899109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:16.899109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:16.899109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:16.899109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:16.899109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:16.899109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:16.899109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:16.899109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:16.899109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:16.899109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:16.899109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:16.900041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:16.900041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:16.900041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:16.900041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:16.900041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:16.900041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:16.900041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:16.900041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:16.900041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:16.900041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:16.900041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:16.900041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:16.900041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:16.900041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:16.900041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:16.900041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:16.900041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:16.900941 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:16.900941 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:16.900941 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:16.900941 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:16.903222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13852c4c-ef54-4f56-9797-aa651c8fe976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:16.904266 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13852c4c-ef54-4f56-9797-aa651c8fe976 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:16.904754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13852c4c-ef54-4f56-9797-aa651c8fe976 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:16.905244 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13852c4c-ef54-4f56-9797-aa651c8fe976 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:16.907169 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 369/726] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:16.915198 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb353e95-ecc5-4c67-8b35-caeaafa0681c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:16.924159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13852c4c-ef54-4f56-9797-aa651c8fe976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:16.944056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-13852c4c-ef54-4f56-9797-aa651c8fe976 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:46:16.944056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:16.944056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:16.944056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:16.944056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:16.944056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:16.944056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:16.944056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:16.944056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:16.944056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:16.944056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:16.944056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:16.944056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:16.944056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:16.944056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:16.944056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:16.945097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:16.945097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:16.945097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:16.945097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:16.945097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:16.945097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:16.945097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:16.945097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:16.945097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:16.945097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:16.945097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:16.945097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:16.945097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:16.945097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:16.945097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:16.945097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:16.945951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:16.945951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:16.945951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:16.945951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:16.945951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:16.945951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:16.945951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:16.945951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:16.945951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:16.945951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:16.945951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:46:16.945951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:16.953227 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 358/727] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:46:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:16.968437 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13852c4c-ef54-4f56-9797-aa651c8fe976 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:17.060033 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:17.094941 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:17.137661 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13852c4c-ef54-4f56-9797-aa651c8fe976 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:17.145919 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb353e95-ecc5-4c67-8b35-caeaafa0681c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:17.186373 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cb762-bad7-4557-a1aa-d1f499cf99bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:17.187143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb762-bad7-4557-a1aa-d1f499cf99bf None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:17.187433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb762-bad7-4557-a1aa-d1f499cf99bf None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:17.187712 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb762-bad7-4557-a1aa-d1f499cf99bf None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:17.188459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2cb762-bad7-4557-a1aa-d1f499cf99bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:17.188773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2cb762-bad7-4557-a1aa-d1f499cf99bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:17.190215 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2cb762-bad7-4557-a1aa-d1f499cf99bf None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:17.203427 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e135df1-7526-4f8d-90e7-149c47c4412a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:17.204121 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e135df1-7526-4f8d-90e7-149c47c4412a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:17.204409 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e135df1-7526-4f8d-90e7-149c47c4412a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:17.204526 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 370/728] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:17.204691 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e135df1-7526-4f8d-90e7-149c47c4412a None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:17.205464 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e135df1-7526-4f8d-90e7-149c47c4412a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:17.205776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e135df1-7526-4f8d-90e7-149c47c4412a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:17.213433 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e135df1-7526-4f8d-90e7-149c47c4412a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:17.214858 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cb762-bad7-4557-a1aa-d1f499cf99bf None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:17.218015 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 359/729] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:46:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:17.225998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e135df1-7526-4f8d-90e7-149c47c4412a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:17.332083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cb762-bad7-4557-a1aa-d1f499cf99bf None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:17.367237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:17.374231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b74cc-dafb-49da-b4a2-0c489a5bfa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:17.375358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b74cc-dafb-49da-b4a2-0c489a5bfa8c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:17.375840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b74cc-dafb-49da-b4a2-0c489a5bfa8c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:17.376296 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b74cc-dafb-49da-b4a2-0c489a5bfa8c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:17.385995 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b74cc-dafb-49da-b4a2-0c489a5bfa8c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:17.387759 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b74cc-dafb-49da-b4a2-0c489a5bfa8c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:17.398898 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e135df1-7526-4f8d-90e7-149c47c4412a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:17.404626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-600b74cc-dafb-49da-b4a2-0c489a5bfa8c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:46:17.404626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:17.404626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:17.404626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:17.404626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:17.404626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:17.404626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:17.404626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:17.404626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:17.404626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:17.404626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:17.404626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:17.404626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:17.404626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:17.404626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:17.404626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:17.405559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:17.406542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:17.406542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:17.406542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:17.406542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:17.406542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:17.406542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:17.406542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:17.406542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:17.406542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:46:17.406542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:17.409263 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 371/730] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Jun 16 18:46:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:17.410871 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e0893-9872-4a45-a162-4446b70a1b8c None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:17.411359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e0893-9872-4a45-a162-4446b70a1b8c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:17.411817 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e0893-9872-4a45-a162-4446b70a1b8c None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:17.414626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 372/731] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:17.415562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e0893-9872-4a45-a162-4446b70a1b8c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:17.434611 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:17.447579 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac9a1c-28dc-4562-8414-0322911a6bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:17.448608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac9a1c-28dc-4562-8414-0322911a6bab None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:17.449082 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac9a1c-28dc-4562-8414-0322911a6bab None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:17.449373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac9a1c-28dc-4562-8414-0322911a6bab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:17.463779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac9a1c-28dc-4562-8414-0322911a6bab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:17.465688 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac9a1c-28dc-4562-8414-0322911a6bab None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:17.487927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-7aac9a1c-28dc-4562-8414-0322911a6bab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:17.487927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:17.487927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:17.487927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:17.487927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:17.487927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:17.487927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:17.487927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:17.487927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:17.487927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:17.487927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:17.487927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:17.487927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:17.487927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:17.487927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:17.488877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:17.488877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:17.488877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:17.488877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:17.488877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:17.488877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:17.488877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:17.488877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:17.488877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:17.488877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:17.488877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:17.488877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:17.488877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:17.488877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:17.488877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:17.488877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:17.488877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:17.489809 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:17.489809 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:17.489809 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:17.489809 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:17.492353 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 360/732] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:17.493984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190a742-6a06-49e4-98cf-7e82423b8ab2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:17.494493 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190a742-6a06-49e4-98cf-7e82423b8ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:17.495020 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190a742-6a06-49e4-98cf-7e82423b8ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:17.512642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9190a742-6a06-49e4-98cf-7e82423b8ab2 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:17.514221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9190a742-6a06-49e4-98cf-7e82423b8ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:17.539523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e0893-9872-4a45-a162-4446b70a1b8c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:17.583748 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfba0e-d6b1-475d-8e3c-9d10acc0c1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:17.584447 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfba0e-d6b1-475d-8e3c-9d10acc0c1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:17.584770 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfba0e-d6b1-475d-8e3c-9d10acc0c1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:17.585049 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfba0e-d6b1-475d-8e3c-9d10acc0c1e0 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:17.585891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccfba0e-d6b1-475d-8e3c-9d10acc0c1e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:17.586200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccfba0e-d6b1-475d-8e3c-9d10acc0c1e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:17.587488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 361/733] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:46:17.587666 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccfba0e-d6b1-475d-8e3c-9d10acc0c1e0 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:17.588687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190a742-6a06-49e4-98cf-7e82423b8ab2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:17.623061 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 373/734] 10.222.0.70 () {56 vars in 1149 bytes} [Thu Jun 16 18:46:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:17.624185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfba0e-d6b1-475d-8e3c-9d10acc0c1e0 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:17.707037 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:17.731507 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190a742-6a06-49e4-98cf-7e82423b8ab2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:17.746967 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:17.765838 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfba0e-d6b1-475d-8e3c-9d10acc0c1e0 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:17.778856 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a9694-9c9d-4f8f-b006-2f50625db046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:17.780007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a9694-9c9d-4f8f-b006-2f50625db046 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:17.780430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a9694-9c9d-4f8f-b006-2f50625db046 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:17.780839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a9694-9c9d-4f8f-b006-2f50625db046 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:17.782425 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a9694-9c9d-4f8f-b006-2f50625db046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:17.782913 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a9694-9c9d-4f8f-b006-2f50625db046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:17.784677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a9694-9c9d-4f8f-b006-2f50625db046 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:17.802112 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 362/735] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:17.803203 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a9694-9c9d-4f8f-b006-2f50625db046 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:17.809845 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39c627-c92e-4e44-8ac6-359cf6b8c512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:17.810649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39c627-c92e-4e44-8ac6-359cf6b8c512 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:17.811005 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39c627-c92e-4e44-8ac6-359cf6b8c512 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:17.811346 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39c627-c92e-4e44-8ac6-359cf6b8c512 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:17.812134 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39c627-c92e-4e44-8ac6-359cf6b8c512 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:17.813820 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39c627-c92e-4e44-8ac6-359cf6b8c512 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:17.838606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 374/736] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Jun 16 18:46:17 2022] POST /identity/v3/services => generated 240 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:46:17.838606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39c627-c92e-4e44-8ac6-359cf6b8c512 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:17.975266 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a9694-9c9d-4f8f-b006-2f50625db046 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:18.064361 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62c1bf-af68-453e-92f5-a462e1e2e769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:18.065319 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62c1bf-af68-453e-92f5-a462e1e2e769 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:18.065746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62c1bf-af68-453e-92f5-a462e1e2e769 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:18.066118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62c1bf-af68-453e-92f5-a462e1e2e769 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:18.069019 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39c627-c92e-4e44-8ac6-359cf6b8c512 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:18.084424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:18.108666 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:18.118685 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ef841-5deb-4a8e-8662-b4f8fc00c70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:18.119805 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ef841-5deb-4a8e-8662-b4f8fc00c70f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:18.120551 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ef841-5deb-4a8e-8662-b4f8fc00c70f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:18.121013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ef841-5deb-4a8e-8662-b4f8fc00c70f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:18.130464 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ef841-5deb-4a8e-8662-b4f8fc00c70f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:18.132645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ef841-5deb-4a8e-8662-b4f8fc00c70f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:18.151334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-4e2ef841-5deb-4a8e-8662-b4f8fc00c70f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:46:18.151334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:18.151334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:18.151334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:18.151334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:18.151334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:18.151334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:18.151334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:18.151334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:18.151334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:18.151334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:18.151334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:18.151334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:18.151334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:18.151334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:18.151334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:18.152769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:18.153719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:18.153719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:18.153719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:18.153719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:18.153719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:18.153719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:18.153719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:18.153719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:18.153719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:46:18.153719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:18.153719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 375/737] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:46:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:18.154300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044c695-2318-40fa-86ab-9db4e571cab0 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:18.155017 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044c695-2318-40fa-86ab-9db4e571cab0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:18.155017 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044c695-2318-40fa-86ab-9db4e571cab0 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:18.156279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 376/738] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:18.164775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044c695-2318-40fa-86ab-9db4e571cab0 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:18.166871 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62c1bf-af68-453e-92f5-a462e1e2e769 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:18.168357 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62c1bf-af68-453e-92f5-a462e1e2e769 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:18.280030 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044c695-2318-40fa-86ab-9db4e571cab0 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:18.286334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 363/739] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:46:17 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:46:18.286334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f76cc-d6b5-4bdd-bf65-9d7acc5be5f9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:18.286334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f76cc-d6b5-4bdd-bf65-9d7acc5be5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:18.286334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f76cc-d6b5-4bdd-bf65-9d7acc5be5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:18.300115 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b55f76cc-d6b5-4bdd-bf65-9d7acc5be5f9 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:18.301285 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b55f76cc-d6b5-4bdd-bf65-9d7acc5be5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:18.322355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f8d49-31e7-4738-bb3d-8cbc382c6adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:18.323070 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f8d49-31e7-4738-bb3d-8cbc382c6adc None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:18.323349 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f8d49-31e7-4738-bb3d-8cbc382c6adc None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:18.323623 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f8d49-31e7-4738-bb3d-8cbc382c6adc None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:18.324459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f8d49-31e7-4738-bb3d-8cbc382c6adc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:18.324767 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f8d49-31e7-4738-bb3d-8cbc382c6adc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:18.326185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f8d49-31e7-4738-bb3d-8cbc382c6adc None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:18.344184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 377/740] 10.222.0.70 () {56 vars in 1135 bytes} [Thu Jun 16 18:46:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:18.345489 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efb8f7-b6c7-4a14-b625-bf7b2f6ef4d7 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:18.345776 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efb8f7-b6c7-4a14-b625-bf7b2f6ef4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:18.346048 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efb8f7-b6c7-4a14-b625-bf7b2f6ef4d7 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:18.350988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 378/741] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:18.350988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7efb8f7-b6c7-4a14-b625-bf7b2f6ef4d7 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:18.375011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:18.375701 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:18.390988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 364/742] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:18 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:46:18.392072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f76cc-d6b5-4bdd-bf65-9d7acc5be5f9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:18.468792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7efb8f7-b6c7-4a14-b625-bf7b2f6ef4d7 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:18.500521 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f76cc-d6b5-4bdd-bf65-9d7acc5be5f9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:18.508045 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3b91a-8a44-4f80-8022-2559c3f76360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:18.509259 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3b91a-8a44-4f80-8022-2559c3f76360 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:18.509542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3b91a-8a44-4f80-8022-2559c3f76360 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:18.509816 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3b91a-8a44-4f80-8022-2559c3f76360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:18.521041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3b91a-8a44-4f80-8022-2559c3f76360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:18.523414 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3b91a-8a44-4f80-8022-2559c3f76360 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:18.538538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab221f1-9a3d-4087-bc7d-c054ce0d130b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:18.539372 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab221f1-9a3d-4087-bc7d-c054ce0d130b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:18.539651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab221f1-9a3d-4087-bc7d-c054ce0d130b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:18.539929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab221f1-9a3d-4087-bc7d-c054ce0d130b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:18.541704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-09a3b91a-8a44-4f80-8022-2559c3f76360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:18.541704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:18.541704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:18.541704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:18.541704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:18.541704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:18.541704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:18.541704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:18.541704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:18.541704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:18.541704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:18.541704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:18.541704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:18.541704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:18.541704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:18.542615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:18.542615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:18.542615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:18.542615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:18.542615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:18.542615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:18.542615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:18.542615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:18.542615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:18.542615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:18.542615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:18.542615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:18.542615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:18.542615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:18.542615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:18.542615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:18.542615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:18.543519 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:18.543519 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:18.543519 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:18.543519 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:18.543519 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 379/743] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:18.544344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89e49d-ea45-40e0-b0da-367053fb2c0f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:18.544567 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89e49d-ea45-40e0-b0da-367053fb2c0f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:18.544841 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89e49d-ea45-40e0-b0da-367053fb2c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:18.562536 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2f89e49d-ea45-40e0-b0da-367053fb2c0f None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:18.563745 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2f89e49d-ea45-40e0-b0da-367053fb2c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:18.611490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:18.624149 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab221f1-9a3d-4087-bc7d-c054ce0d130b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:18.625619 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab221f1-9a3d-4087-bc7d-c054ce0d130b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:18.643256 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 380/744] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:18 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:46:18.644547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89e49d-ea45-40e0-b0da-367053fb2c0f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:18.663484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:18.749806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 365/745] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:46:18 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:46:18.751043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab221f1-9a3d-4087-bc7d-c054ce0d130b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:18.777772 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89e49d-ea45-40e0-b0da-367053fb2c0f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:18.813560 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4094b-aa94-4425-9e33-ed78987c3b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:18.814150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4094b-aa94-4425-9e33-ed78987c3b14 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:18.814429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4094b-aa94-4425-9e33-ed78987c3b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:18.814740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4094b-aa94-4425-9e33-ed78987c3b14 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:18.815456 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4094b-aa94-4425-9e33-ed78987c3b14 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:18.816831 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4094b-aa94-4425-9e33-ed78987c3b14 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:18.831317 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 381/746] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Jun 16 18:46:18 2022] POST /identity/v3/services => generated 251 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:46:18.832126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4094b-aa94-4425-9e33-ed78987c3b14 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:18.882931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:18.895545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab221f1-9a3d-4087-bc7d-c054ce0d130b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:18.914500 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:18.962391 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d5ae4e-57b6-4a22-8044-27304b5e332c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:18.963284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5ae4e-57b6-4a22-8044-27304b5e332c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:18.963651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5ae4e-57b6-4a22-8044-27304b5e332c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:18.964001 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5ae4e-57b6-4a22-8044-27304b5e332c None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:18.964815 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d5ae4e-57b6-4a22-8044-27304b5e332c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:18.965195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d5ae4e-57b6-4a22-8044-27304b5e332c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:18.966548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4094b-aa94-4425-9e33-ed78987c3b14 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:18.966912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d5ae4e-57b6-4a22-8044-27304b5e332c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:18.984784 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 366/747] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:18.984784 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d5ae4e-57b6-4a22-8044-27304b5e332c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:19.010679 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf667a8e-e3fc-4241-8bb2-5d0fa79a3bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:19.011796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf667a8e-e3fc-4241-8bb2-5d0fa79a3bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:19.012223 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf667a8e-e3fc-4241-8bb2-5d0fa79a3bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:19.012676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf667a8e-e3fc-4241-8bb2-5d0fa79a3bd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:19.026061 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf667a8e-e3fc-4241-8bb2-5d0fa79a3bd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:19.034630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-bf667a8e-e3fc-4241-8bb2-5d0fa79a3bd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:46:19.034630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:19.034630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:19.034630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:19.034630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:19.034630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:19.034630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:19.034630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:19.034630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:19.034630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:19.034630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:19.034630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:19.034630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:19.034630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:19.034630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:19.034630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:19.035645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:19.035645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:19.035645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:19.035645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:19.035645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:19.035645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:19.035645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:19.035645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:19.035645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:19.035645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:19.035645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:19.035645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:19.035645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:19.035645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:19.035645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:19.035645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:19.036498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:19.036498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:19.036498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:19.036498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:19.036498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:19.036498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:19.036498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:19.036498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:19.036498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:19.036498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:19.036498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:46:19.036498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:19.036498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 382/748] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:46:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:19.037203 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf667a8e-e3fc-4241-8bb2-5d0fa79a3bd9 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:19.099910 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d5ae4e-57b6-4a22-8044-27304b5e332c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:19.139647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac621f37-f866-46f4-915c-1ff8eae0646d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:19.140413 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac621f37-f866-46f4-915c-1ff8eae0646d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:19.140694 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac621f37-f866-46f4-915c-1ff8eae0646d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:19.140962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac621f37-f866-46f4-915c-1ff8eae0646d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:19.154941 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac621f37-f866-46f4-915c-1ff8eae0646d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:19.156563 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac621f37-f866-46f4-915c-1ff8eae0646d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:19.162316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf667a8e-e3fc-4241-8bb2-5d0fa79a3bd9 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:19.163571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:19.175847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ac621f37-f866-46f4-915c-1ff8eae0646d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:19.175847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:19.175847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:19.175847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:19.175847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:19.175847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:19.175847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:19.175847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:19.175847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:19.175847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:19.175847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:19.175847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:19.175847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:19.175847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:19.175847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:19.176785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:19.176785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:19.176785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:19.176785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:19.176785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:19.176785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:19.176785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:19.176785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:19.176785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:19.176785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:19.176785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:19.176785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:19.176785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:19.176785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:19.176785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:19.176785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:19.176785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:19.177697 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:19.177697 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:19.177697 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:19.177697 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:19.177697 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 367/749] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:19.178631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac621f37-f866-46f4-915c-1ff8eae0646d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:19.193291 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:19.205851 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4579a2-acd3-4a28-927e-5dde9dd82a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:19.206826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4579a2-acd3-4a28-927e-5dde9dd82a2b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:19.207353 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4579a2-acd3-4a28-927e-5dde9dd82a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:19.207725 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4579a2-acd3-4a28-927e-5dde9dd82a2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:19.216125 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4579a2-acd3-4a28-927e-5dde9dd82a2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:19.217704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4579a2-acd3-4a28-927e-5dde9dd82a2b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:19.234311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-3f4579a2-acd3-4a28-927e-5dde9dd82a2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:19.234311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:19.234311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:19.234311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:19.234311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:19.234311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:19.234311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:19.234311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:19.234311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:19.234311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:19.234311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:19.234311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:19.234311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:19.234311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:19.234311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:19.235301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:19.235301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:19.235301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:19.235301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:19.235301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:19.235301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:19.235301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:19.235301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:19.235301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:19.235301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:19.235301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:19.235301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:19.235301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:19.235301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:19.235301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:19.235301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:19.235301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:19.236182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:19.236182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:19.236182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:19.236182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:19.236182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:19.236182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:19.236182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:19.236182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:19.236182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 383/750] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:19.237684 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4579a2-acd3-4a28-927e-5dde9dd82a2b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:19.291985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac621f37-f866-46f4-915c-1ff8eae0646d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:19.327222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9291be-7ba4-4f31-a94c-3cc00d0b6920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:19.328127 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9291be-7ba4-4f31-a94c-3cc00d0b6920 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:19.328548 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9291be-7ba4-4f31-a94c-3cc00d0b6920 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:19.328959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9291be-7ba4-4f31-a94c-3cc00d0b6920 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:19.329866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9291be-7ba4-4f31-a94c-3cc00d0b6920 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:19.330312 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9291be-7ba4-4f31-a94c-3cc00d0b6920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:19.331819 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9291be-7ba4-4f31-a94c-3cc00d0b6920 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:19.341175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 368/751] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:46:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:19.342316 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9291be-7ba4-4f31-a94c-3cc00d0b6920 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:19.352747 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4579a2-acd3-4a28-927e-5dde9dd82a2b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:19.390379 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3db521-c2e0-4757-ab19-8570be31a27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:19.391105 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3db521-c2e0-4757-ab19-8570be31a27b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:19.391390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3db521-c2e0-4757-ab19-8570be31a27b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:19.391665 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3db521-c2e0-4757-ab19-8570be31a27b None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:19.393065 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3db521-c2e0-4757-ab19-8570be31a27b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:19.393376 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3db521-c2e0-4757-ab19-8570be31a27b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:19.394844 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3db521-c2e0-4757-ab19-8570be31a27b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:19.408188 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 384/752] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:19.409124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3db521-c2e0-4757-ab19-8570be31a27b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:19.410333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:19.462805 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9291be-7ba4-4f31-a94c-3cc00d0b6920 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:19.466968 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:19.498866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c989f6-f2ac-4a97-82af-d86f7592f0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:19.499534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c989f6-f2ac-4a97-82af-d86f7592f0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:19.499817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c989f6-f2ac-4a97-82af-d86f7592f0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:19.500096 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c989f6-f2ac-4a97-82af-d86f7592f0b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:19.500887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c989f6-f2ac-4a97-82af-d86f7592f0b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:19.501199 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c989f6-f2ac-4a97-82af-d86f7592f0b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:19.502647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c989f6-f2ac-4a97-82af-d86f7592f0b1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:19.522674 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 369/753] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:19.522674 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c989f6-f2ac-4a97-82af-d86f7592f0b1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:19.550176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3db521-c2e0-4757-ab19-8570be31a27b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:19.587790 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87331b-1e88-4ad2-a3a0-3d64cffbe975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:19.588828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87331b-1e88-4ad2-a3a0-3d64cffbe975 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:19.589255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87331b-1e88-4ad2-a3a0-3d64cffbe975 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:19.589713 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87331b-1e88-4ad2-a3a0-3d64cffbe975 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:19.601301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87331b-1e88-4ad2-a3a0-3d64cffbe975 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:19.603121 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87331b-1e88-4ad2-a3a0-3d64cffbe975 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:19.622315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-2c87331b-1e88-4ad2-a3a0-3d64cffbe975 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:19.622315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:19.622315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:19.622315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:19.622315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:19.622315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:19.622315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:19.622315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:19.622315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:19.622315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:19.622315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:19.622315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:19.622315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:19.622315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:19.622315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:19.623485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:19.623485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:19.623485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:19.623485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:19.623485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:19.623485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:19.623485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:19.623485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:19.623485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:19.623485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:19.623485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:19.623485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:19.623485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:19.623485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:19.623485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:19.623485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:19.623485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:19.624411 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:19.624411 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:19.624411 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:19.624411 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:19.630783 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 385/754] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:19.632161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87331b-1e88-4ad2-a3a0-3d64cffbe975 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:19.651557 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c989f6-f2ac-4a97-82af-d86f7592f0b1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:19.680043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:19.697392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda60a7-81f0-45f6-9694-9c05aad93df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:19.698555 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda60a7-81f0-45f6-9694-9c05aad93df0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:19.699041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda60a7-81f0-45f6-9694-9c05aad93df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:19.699909 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda60a7-81f0-45f6-9694-9c05aad93df0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:19.710606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda60a7-81f0-45f6-9694-9c05aad93df0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:19.712635 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda60a7-81f0-45f6-9694-9c05aad93df0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:19.731042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-8fda60a7-81f0-45f6-9694-9c05aad93df0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:19.731042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:19.731042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:19.731042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:19.731042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:19.731042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:19.731042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:19.731042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:19.731042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:19.731042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:19.731042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:19.731042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:19.731042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:19.731042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:19.731042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:19.731998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:19.731998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:19.731998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:19.731998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:19.731998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:19.731998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:19.731998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:19.731998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:19.731998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:19.731998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:19.731998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:19.731998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:19.731998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:19.731998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:19.731998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:19.731998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:19.731998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:19.732925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:19.732925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:19.732925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:19.732925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:19.732925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:19.732925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:19.732925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:19.732925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:19.735483 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 370/755] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:19.736433 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda60a7-81f0-45f6-9694-9c05aad93df0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:19.757874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:19.770873 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87331b-1e88-4ad2-a3a0-3d64cffbe975 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:19.814869 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8b802-f9f6-41d8-8638-e3e3b7fe6de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:19.815861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8b802-f9f6-41d8-8638-e3e3b7fe6de4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:19.816316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8b802-f9f6-41d8-8638-e3e3b7fe6de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:19.816766 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8b802-f9f6-41d8-8638-e3e3b7fe6de4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:19.817777 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8b802-f9f6-41d8-8638-e3e3b7fe6de4 None admin] RBAC: Adding query filter params (role.id=c123034c11cc4d9ea6cab00e05bdcd53, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=02c826fc0baa480393df4c884c9e35dc) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:19.818285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8b802-f9f6-41d8-8638-e3e3b7fe6de4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:19.820500 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8b802-f9f6-41d8-8638-e3e3b7fe6de4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:19.827046 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 386/756] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:46:19 2022] GET /identity/v3/role_assignments?user.id=02c826fc0baa480393df4c884c9e35dc&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=c123034c11cc4d9ea6cab00e05bdcd53 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:19.828026 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8b802-f9f6-41d8-8638-e3e3b7fe6de4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:19.900283 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda60a7-81f0-45f6-9694-9c05aad93df0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:19.988941 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8b802-f9f6-41d8-8638-e3e3b7fe6de4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:19.994314 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14567a0-555c-4670-9155-cd8d8c299284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:19.995035 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14567a0-555c-4670-9155-cd8d8c299284 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:19.995317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14567a0-555c-4670-9155-cd8d8c299284 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:19.995591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14567a0-555c-4670-9155-cd8d8c299284 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:19.996323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14567a0-555c-4670-9155-cd8d8c299284 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:19.996627 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14567a0-555c-4670-9155-cd8d8c299284 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:19.998082 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14567a0-555c-4670-9155-cd8d8c299284 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:20.027541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 371/757] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:20.033487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14567a0-555c-4670-9155-cd8d8c299284 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:20.052654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364972b-1a2b-463f-bc32-bd2dd5a1d5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:20.053618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364972b-1a2b-463f-bc32-bd2dd5a1d5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:20.053935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364972b-1a2b-463f-bc32-bd2dd5a1d5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:20.054332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364972b-1a2b-463f-bc32-bd2dd5a1d5f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:20.055218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364972b-1a2b-463f-bc32-bd2dd5a1d5f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:20.055539 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364972b-1a2b-463f-bc32-bd2dd5a1d5f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:20.057083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364972b-1a2b-463f-bc32-bd2dd5a1d5f2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:20.074396 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 387/758] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:20.079926 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364972b-1a2b-463f-bc32-bd2dd5a1d5f2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:20.104630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:20.108049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:20.194524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364972b-1a2b-463f-bc32-bd2dd5a1d5f2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:20.237206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1b7fa-1d64-4821-858f-9f0e994261f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:20.237866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1b7fa-1d64-4821-858f-9f0e994261f3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:20.238143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1b7fa-1d64-4821-858f-9f0e994261f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:20.238415 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1b7fa-1d64-4821-858f-9f0e994261f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:20.239275 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e1b7fa-1d64-4821-858f-9f0e994261f3 None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=7b22fe0ab0354385924923d0d443acae) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:20.239613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e1b7fa-1d64-4821-858f-9f0e994261f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:20.241858 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e1b7fa-1d64-4821-858f-9f0e994261f3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:20.248267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 388/759] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:46:20 2022] GET /identity/v3/role_assignments?user.id=7b22fe0ab0354385924923d0d443acae&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:20.257860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14567a0-555c-4670-9155-cd8d8c299284 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:20.262603 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd4fc5-f585-49cf-9611-a50440a453bf None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:20.262603 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd4fc5-f585-49cf-9611-a50440a453bf None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:20.262603 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd4fc5-f585-49cf-9611-a50440a453bf None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:20.263238 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 389/760] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:46:20.269257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62970c8-44b5-4e77-9465-57a55dd8010a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:20.269591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62970c8-44b5-4e77-9465-57a55dd8010a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:20.269808 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62970c8-44b5-4e77-9465-57a55dd8010a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:20.289043 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-a62970c8-44b5-4e77-9465-57a55dd8010a None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:20.290668 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a62970c8-44b5-4e77-9465-57a55dd8010a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:20.339267 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b18e5-3d44-4dde-b14c-09e355274961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:20.340903 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b18e5-3d44-4dde-b14c-09e355274961 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:20.341393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b18e5-3d44-4dde-b14c-09e355274961 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:20.341909 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b18e5-3d44-4dde-b14c-09e355274961 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:20.350564 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b18e5-3d44-4dde-b14c-09e355274961 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:20.352549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b18e5-3d44-4dde-b14c-09e355274961 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:20.374704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-9d2b18e5-3d44-4dde-b14c-09e355274961 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:46:20.374704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:20.374704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:20.374704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:20.374704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:20.374704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:20.374704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:20.374704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:20.374704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:20.374704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:20.374704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:20.374704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:20.374704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:20.374704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:20.374704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:20.374704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:20.375707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:20.376695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:20.376695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:20.376695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:20.376695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:20.376695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:20.376695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:20.376695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:20.376695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:20.376695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:46:20.376695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:20.376695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 372/761] 10.222.0.70 () {56 vars in 1093 bytes} [Thu Jun 16 18:46:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:20.380262 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 390/762] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:46:20.383666 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b18e5-3d44-4dde-b14c-09e355274961 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:20.407384 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62970c8-44b5-4e77-9465-57a55dd8010a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:20.425178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:20.509041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:20.512452 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b18e5-3d44-4dde-b14c-09e355274961 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:20.545523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62970c8-44b5-4e77-9465-57a55dd8010a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:20.578293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef7449b-810c-490b-bef9-88d46629e21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:20.579003 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7449b-810c-490b-bef9-88d46629e21e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:20.579282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7449b-810c-490b-bef9-88d46629e21e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:20.579558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7449b-810c-490b-bef9-88d46629e21e None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:20.580393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef7449b-810c-490b-bef9-88d46629e21e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:20.580698 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef7449b-810c-490b-bef9-88d46629e21e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:20.582118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef7449b-810c-490b-bef9-88d46629e21e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:20.615239 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c3fc0-041a-4c21-be94-a2ba4586d28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:20.616151 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c3fc0-041a-4c21-be94-a2ba4586d28b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:20.616475 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c3fc0-041a-4c21-be94-a2ba4586d28b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:20.616762 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c3fc0-041a-4c21-be94-a2ba4586d28b None admin] PATH_INFO: `/v3/services/network` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:20.617423 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c3fc0-041a-4c21-be94-a2ba4586d28b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:20.618888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c3fc0-041a-4c21-be94-a2ba4586d28b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:20.621398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 373/763] 10.222.0.70 () {56 vars in 1133 bytes} [Thu Jun 16 18:46:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:20.626725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef7449b-810c-490b-bef9-88d46629e21e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:20.648080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-0a4c3fc0-041a-4c21-be94-a2ba4586d28b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:46:20.648080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:20.648080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:20.648080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:20.648080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:20.648080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:20.648080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:20.648080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:20.648080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:46:20.648080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:20.648080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:20.648080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:20.648080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:20.648080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:20.648080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:20.648080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:20.648080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:20.651195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:20.651195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:20.651195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:20.651195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:20.651195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:20.651195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:20.651195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:20.651195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:20.651195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:20.651195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:20.651195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:20.651195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:20.651195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:20.651195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:20.651195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:20.651195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:20.653882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:20.653882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:20.653882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:20.653882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:20.653882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:20.653882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:20.653882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:20.653882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:20.653882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:20.653882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:20.653882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:20.653882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:20.653882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:46:20.653882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:20.653882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 391/764] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:20 2022] GET /identity/v3/services/network => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:20.662931 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c3fc0-041a-4c21-be94-a2ba4586d28b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:20.802626 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef7449b-810c-490b-bef9-88d46629e21e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:20.815130 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:20.837404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c3fc0-041a-4c21-be94-a2ba4586d28b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:20.878267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed229c9-9157-4102-9e5f-4135cd6b5767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:20.881681 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed229c9-9157-4102-9e5f-4135cd6b5767 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:20.881681 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed229c9-9157-4102-9e5f-4135cd6b5767 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:20.882157 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49027102-b052-4f25-9306-fbe7a7b93097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:20.882344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed229c9-9157-4102-9e5f-4135cd6b5767 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:20.882494 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49027102-b052-4f25-9306-fbe7a7b93097 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:20.882666 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed229c9-9157-4102-9e5f-4135cd6b5767 None admin] RBAC: Adding query filter params (name=network) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:20.882866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49027102-b052-4f25-9306-fbe7a7b93097 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:20.883013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed229c9-9157-4102-9e5f-4135cd6b5767 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:20.883162 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49027102-b052-4f25-9306-fbe7a7b93097 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:20.883316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed229c9-9157-4102-9e5f-4135cd6b5767 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:20.893503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 392/765] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Jun 16 18:46:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:20.899193 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed229c9-9157-4102-9e5f-4135cd6b5767 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:20.906123 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49027102-b052-4f25-9306-fbe7a7b93097 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:20.912618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49027102-b052-4f25-9306-fbe7a7b93097 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:20.942400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-49027102-b052-4f25-9306-fbe7a7b93097 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:20.942400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:20.942400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:20.942400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:20.942400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:20.942400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:20.942400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:20.942400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:20.942400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:20.942400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:20.942400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:20.942400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:20.942400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:20.942400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:20.942400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:20.943423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:20.943423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:20.943423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:20.943423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:20.943423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:20.943423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:20.943423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:20.943423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:20.943423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:20.943423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:20.943423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:20.943423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:20.943423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:20.943423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:20.943423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:20.943423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:20.943423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:20.944887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:20.944887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:20.944887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:20.944887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:20.950617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 374/766] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:20.957234 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49027102-b052-4f25-9306-fbe7a7b93097 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:20.986932 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:21.088579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed229c9-9157-4102-9e5f-4135cd6b5767 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:21.139403 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49027102-b052-4f25-9306-fbe7a7b93097 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:21.160842 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d72aa0-8b5b-4a43-9acb-fd70e84b5502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:21.161517 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d72aa0-8b5b-4a43-9acb-fd70e84b5502 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:21.162185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d72aa0-8b5b-4a43-9acb-fd70e84b5502 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:21.162486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d72aa0-8b5b-4a43-9acb-fd70e84b5502 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:21.163789 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d72aa0-8b5b-4a43-9acb-fd70e84b5502 None admin] RBAC: Adding query filter params (type=network) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:21.164204 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d72aa0-8b5b-4a43-9acb-fd70e84b5502 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:21.165568 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d72aa0-8b5b-4a43-9acb-fd70e84b5502 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:21.178438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 393/767] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Jun 16 18:46:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:21.181828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d72aa0-8b5b-4a43-9acb-fd70e84b5502 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:21.195556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:21.210174 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f7d09-1d0a-44a9-92d7-aa44cc42ec5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:21.210936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f7d09-1d0a-44a9-92d7-aa44cc42ec5c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:21.211208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f7d09-1d0a-44a9-92d7-aa44cc42ec5c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:21.211487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f7d09-1d0a-44a9-92d7-aa44cc42ec5c None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:21.212218 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f7d09-1d0a-44a9-92d7-aa44cc42ec5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:21.213839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f7d09-1d0a-44a9-92d7-aa44cc42ec5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:21.216185 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f7d09-1d0a-44a9-92d7-aa44cc42ec5c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:21.238943 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 375/768] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:21.248725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f7d09-1d0a-44a9-92d7-aa44cc42ec5c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:21.362467 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d72aa0-8b5b-4a43-9acb-fd70e84b5502 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:21.390921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:21.403980 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f2327-4f09-4e76-9f84-b151aabd8987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:21.404775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f2327-4f09-4e76-9f84-b151aabd8987 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:21.405064 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f2327-4f09-4e76-9f84-b151aabd8987 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:21.405339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f2327-4f09-4e76-9f84-b151aabd8987 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:21.406121 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f2327-4f09-4e76-9f84-b151aabd8987 None admin] RBAC: Adding query filter params (interface=public, service_id=28507288d44640e19c6d9fed2afef4ba, region_id=RegionOne) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:21.406430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f2327-4f09-4e76-9f84-b151aabd8987 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:21.407810 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f2327-4f09-4e76-9f84-b151aabd8987 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:21.414983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f7d09-1d0a-44a9-92d7-aa44cc42ec5c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:21.416237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 394/769] 10.222.0.70 () {56 vars in 1251 bytes} [Thu Jun 16 18:46:21 2022] GET /identity/v3/endpoints?service_id=28507288d44640e19c6d9fed2afef4ba&interface=public®ion_id=RegionOne => generated 194 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:21.418422 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841f97b-c5a6-4390-b894-0ef0dca060df None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:21.418739 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841f97b-c5a6-4390-b894-0ef0dca060df None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:21.419015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841f97b-c5a6-4390-b894-0ef0dca060df None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:21.422710 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 395/770] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:21.423525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841f97b-c5a6-4390-b894-0ef0dca060df None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:21.454051 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910dfd30-3eec-4b85-ac0a-a021c3a23a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:21.458617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910dfd30-3eec-4b85-ac0a-a021c3a23a42 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:21.458617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910dfd30-3eec-4b85-ac0a-a021c3a23a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:21.458617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910dfd30-3eec-4b85-ac0a-a021c3a23a42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:21.462817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910dfd30-3eec-4b85-ac0a-a021c3a23a42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:21.464239 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910dfd30-3eec-4b85-ac0a-a021c3a23a42 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:21.479778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-910dfd30-3eec-4b85-ac0a-a021c3a23a42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:21.479778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:21.479778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:21.479778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:21.479778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:21.479778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:21.479778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:21.479778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:21.479778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:21.479778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:21.479778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:21.479778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:21.479778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:21.479778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:21.479778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:21.480868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:21.480868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:21.480868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:21.480868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:21.480868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:21.480868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:21.480868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:21.480868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:21.480868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:21.480868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:21.480868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:21.480868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:21.480868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:21.480868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:21.480868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:21.480868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:21.480868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:21.482042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:21.482042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:21.482042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:21.482042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:21.482042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:21.482042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:21.482042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:21.482042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:21.485618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 376/771] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:21.486997 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a5e6c-cfc6-47cf-a3fb-542e9636c848 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:21.487283 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a5e6c-cfc6-47cf-a3fb-542e9636c848 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:21.487564 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a5e6c-cfc6-47cf-a3fb-542e9636c848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:21.504643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f06a5e6c-cfc6-47cf-a3fb-542e9636c848 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:21.505941 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f06a5e6c-cfc6-47cf-a3fb-542e9636c848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:21.517953 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:21.555408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841f97b-c5a6-4390-b894-0ef0dca060df None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:21.587025 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 377/772] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:21.588178 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a5e6c-cfc6-47cf-a3fb-542e9636c848 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:21.593319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36790ab-09f8-4a31-82c1-4b0704a4df71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:21.594217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36790ab-09f8-4a31-82c1-4b0704a4df71 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:21.594686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36790ab-09f8-4a31-82c1-4b0704a4df71 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:21.595130 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36790ab-09f8-4a31-82c1-4b0704a4df71 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:21.596031 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36790ab-09f8-4a31-82c1-4b0704a4df71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:21.597691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36790ab-09f8-4a31-82c1-4b0704a4df71 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:21.610750 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 396/773] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Jun 16 18:46:21 2022] GET /identity/v3/services => generated 772 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:21.637779 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36790ab-09f8-4a31-82c1-4b0704a4df71 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:21.672818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:21.720018 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a5e6c-cfc6-47cf-a3fb-542e9636c848 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:21.759938 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9977c1-f6bd-4af6-81a1-0393f21d5c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:21.760671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9977c1-f6bd-4af6-81a1-0393f21d5c28 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:21.760955 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9977c1-f6bd-4af6-81a1-0393f21d5c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:21.761274 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9977c1-f6bd-4af6-81a1-0393f21d5c28 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:21.762075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9977c1-f6bd-4af6-81a1-0393f21d5c28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:21.762383 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9977c1-f6bd-4af6-81a1-0393f21d5c28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:21.763856 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9977c1-f6bd-4af6-81a1-0393f21d5c28 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:21.778506 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 378/774] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:21.784518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9977c1-f6bd-4af6-81a1-0393f21d5c28 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:21.869656 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36790ab-09f8-4a31-82c1-4b0704a4df71 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:21.897719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9977c1-f6bd-4af6-81a1-0393f21d5c28 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:21.909257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498c4ea-cc65-4226-94c9-726413dc03b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:21.910072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498c4ea-cc65-4226-94c9-726413dc03b4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:21.910354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498c4ea-cc65-4226-94c9-726413dc03b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:21.911695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498c4ea-cc65-4226-94c9-726413dc03b4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:21.912584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1498c4ea-cc65-4226-94c9-726413dc03b4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:21.914060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1498c4ea-cc65-4226-94c9-726413dc03b4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:21.916135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:21.921193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:21.930420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-1498c4ea-cc65-4226-94c9-726413dc03b4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:46:21.930420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:21.930420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:21.930420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:21.930420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:21.930420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:21.930420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:21.930420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:21.930420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:46:21.930420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:21.930420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:21.930420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:21.930420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:21.930420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:21.930420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:21.930420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:21.931591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:21.931591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:21.931591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:21.931591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:21.931591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:21.931591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:21.931591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:21.931591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:21.931591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:21.931591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:21.931591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:21.931591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:21.931591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:21.931591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:21.931591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:21.931591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:21.932613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:21.932613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:21.932613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:21.932613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:21.932613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:21.932613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:21.932613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:21.932613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:21.932613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:21.932613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:21.932613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:21.932613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:21.932613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:21.932613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:46:21.932613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:21.935161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 397/775] 10.222.0.70 () {56 vars in 1129 bytes} [Thu Jun 16 18:46:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:21.942234 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498c4ea-cc65-4226-94c9-726413dc03b4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:21.942416 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106a025-65e9-47af-8e02-0657eb28caf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:21.947041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106a025-65e9-47af-8e02-0657eb28caf3 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:21.947325 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106a025-65e9-47af-8e02-0657eb28caf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:21.947603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106a025-65e9-47af-8e02-0657eb28caf3 None admin] PATH_INFO: `/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/8342da838fe4462b9de2817ad3ac4ff7/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:21.949744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106a025-65e9-47af-8e02-0657eb28caf3 None admin] RBAC: Authorizing `identity:create_grant(project_id=34185125e75d4aad8ad5382ab22ee8a2, user_id=8342da838fe4462b9de2817ad3ac4ff7, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:21.951231 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106a025-65e9-47af-8e02-0657eb28caf3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:21.977095 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 379/776] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:46:21 2022] PUT /identity/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/8342da838fe4462b9de2817ad3ac4ff7/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:46:22.091807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498c4ea-cc65-4226-94c9-726413dc03b4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:22.145813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40001cdc-bdec-4b0c-a600-b4982151fef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:22.146488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40001cdc-bdec-4b0c-a600-b4982151fef6 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:22.146877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40001cdc-bdec-4b0c-a600-b4982151fef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:22.147357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40001cdc-bdec-4b0c-a600-b4982151fef6 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:22.148190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40001cdc-bdec-4b0c-a600-b4982151fef6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:22.148444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40001cdc-bdec-4b0c-a600-b4982151fef6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:22.149827 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40001cdc-bdec-4b0c-a600-b4982151fef6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:22.159329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 398/777] 10.222.0.70 () {56 vars in 1124 bytes} [Thu Jun 16 18:46:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:22.165752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106a025-65e9-47af-8e02-0657eb28caf3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:22.270880 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79ad9c-62bf-4853-8c0a-9f4023d5a164 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:22.271391 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79ad9c-62bf-4853-8c0a-9f4023d5a164 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:22.271469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79ad9c-62bf-4853-8c0a-9f4023d5a164 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:22.273576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 399/778] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:22.281620 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd7acc-6596-49f5-a1cf-2a1411eecf73 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:22.281908 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd7acc-6596-49f5-a1cf-2a1411eecf73 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:22.282183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd7acc-6596-49f5-a1cf-2a1411eecf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:22.309973 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-04bd7acc-6596-49f5-a1cf-2a1411eecf73 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:22.311216 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-04bd7acc-6596-49f5-a1cf-2a1411eecf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:22.320596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106a025-65e9-47af-8e02-0657eb28caf3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:22.359812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5effab5-d650-4f30-9c2b-4e23ecbf561d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:22.361459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5effab5-d650-4f30-9c2b-4e23ecbf561d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:22.361742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5effab5-d650-4f30-9c2b-4e23ecbf561d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:22.362973 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5effab5-d650-4f30-9c2b-4e23ecbf561d None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:22.362973 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5effab5-d650-4f30-9c2b-4e23ecbf561d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:22.363188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5effab5-d650-4f30-9c2b-4e23ecbf561d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:22.366497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5effab5-d650-4f30-9c2b-4e23ecbf561d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:22.383126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 380/779] 10.222.0.70 () {56 vars in 1124 bytes} [Thu Jun 16 18:46:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:22.387821 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5effab5-d650-4f30-9c2b-4e23ecbf561d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:22.452602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:22.477080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 400/780] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:46:22.502420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:22.522672 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd7acc-6596-49f5-a1cf-2a1411eecf73 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:22.626980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5effab5-d650-4f30-9c2b-4e23ecbf561d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:22.788071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c132b9-4086-4feb-a35c-6281746e09d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:22.788889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c132b9-4086-4feb-a35c-6281746e09d7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:22.789285 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c132b9-4086-4feb-a35c-6281746e09d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:22.789662 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c132b9-4086-4feb-a35c-6281746e09d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:22.790598 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c132b9-4086-4feb-a35c-6281746e09d7 None admin] RBAC: Adding query filter params (interface=public, service_id=259eb7c236ba4f8d9efe4f35f500bbac, region_id=RegionOne) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:22.791015 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c132b9-4086-4feb-a35c-6281746e09d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:22.791855 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd7acc-6596-49f5-a1cf-2a1411eecf73 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:22.795677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c132b9-4086-4feb-a35c-6281746e09d7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:22.811184 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 381/781] 10.222.0.70 () {56 vars in 1251 bytes} [Thu Jun 16 18:46:22 2022] GET /identity/v3/endpoints?service_id=259eb7c236ba4f8d9efe4f35f500bbac&interface=public®ion_id=RegionOne => generated 194 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:22.819646 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c132b9-4086-4feb-a35c-6281746e09d7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:22.840139 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2873fd-223b-4b18-97a9-64e9097beed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:22.841704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2873fd-223b-4b18-97a9-64e9097beed7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:22.842155 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2873fd-223b-4b18-97a9-64e9097beed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:22.842590 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2873fd-223b-4b18-97a9-64e9097beed7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:22.865354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2873fd-223b-4b18-97a9-64e9097beed7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:22.879773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-9a2873fd-223b-4b18-97a9-64e9097beed7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:22.879773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:22.879773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:22.879773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:22.879773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:22.879773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:22.879773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:22.879773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:22.879773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:22.879773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:22.879773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:22.879773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:22.879773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:22.879773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:22.879773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:22.879773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:22.880742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:22.880742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:22.880742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:22.880742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:22.880742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:22.880742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:22.880742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:22.880742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:22.880742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:22.880742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:22.880742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:22.880742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:22.880742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:22.880742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:22.880742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:22.880742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:22.882137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:22.882137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:22.882137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:22.882137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:22.882137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:22.882137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:22.882137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:22.882137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:22.882137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:22.882137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:22.882137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:22.882137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:22.884714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 401/782] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:46:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:22.893118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2873fd-223b-4b18-97a9-64e9097beed7 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:23.067564 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c132b9-4086-4feb-a35c-6281746e09d7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:23.079235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:23.097277 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:23.117869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87212a2-40d2-4ee2-a212-3d434aa15310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:23.118546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87212a2-40d2-4ee2-a212-3d434aa15310 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:23.118854 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87212a2-40d2-4ee2-a212-3d434aa15310 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:23.119128 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87212a2-40d2-4ee2-a212-3d434aa15310 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:23.120105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87212a2-40d2-4ee2-a212-3d434aa15310 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:23.121809 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87212a2-40d2-4ee2-a212-3d434aa15310 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:23.130656 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 382/783] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Jun 16 18:46:22 2022] GET /identity/v3/services => generated 772 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:23.132802 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61659c74-a48f-4b49-b0e2-bcb7200e8277 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:23.133395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61659c74-a48f-4b49-b0e2-bcb7200e8277 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:23.133874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61659c74-a48f-4b49-b0e2-bcb7200e8277 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:23.137199 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 383/784] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:23.143482 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cc868-0d7a-43a0-8cf2-2b21f6b97631 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:23.144919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cc868-0d7a-43a0-8cf2-2b21f6b97631 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:23.145605 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cc868-0d7a-43a0-8cf2-2b21f6b97631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:23.180127 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4b3cc868-0d7a-43a0-8cf2-2b21f6b97631 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:23.182313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4b3cc868-0d7a-43a0-8cf2-2b21f6b97631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:23.184588 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2873fd-223b-4b18-97a9-64e9097beed7 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:23.254123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a4879-15a7-4de5-80ae-b66f6a28e02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:23.255726 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a4879-15a7-4de5-80ae-b66f6a28e02c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:23.256782 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a4879-15a7-4de5-80ae-b66f6a28e02c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:23.258326 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a4879-15a7-4de5-80ae-b66f6a28e02c None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:23.259966 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a4879-15a7-4de5-80ae-b66f6a28e02c None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:23.261181 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a4879-15a7-4de5-80ae-b66f6a28e02c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:23.262828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a4879-15a7-4de5-80ae-b66f6a28e02c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:23.282013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 402/785] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:46:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:23.288166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a4879-15a7-4de5-80ae-b66f6a28e02c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:23.336192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 384/786] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:23.367458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3cc868-0d7a-43a0-8cf2-2b21f6b97631 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:23.426549 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a4879-15a7-4de5-80ae-b66f6a28e02c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:23.443306 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:23.469912 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7366ce3-f665-4daf-8e18-9730eb836e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:23.470724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7366ce3-f665-4daf-8e18-9730eb836e44 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:23.471001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7366ce3-f665-4daf-8e18-9730eb836e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:23.471277 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7366ce3-f665-4daf-8e18-9730eb836e44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:23.481178 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3cc868-0d7a-43a0-8cf2-2b21f6b97631 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:23.481804 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7366ce3-f665-4daf-8e18-9730eb836e44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:23.482778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:23.484412 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7366ce3-f665-4daf-8e18-9730eb836e44 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:23.504640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-c7366ce3-f665-4daf-8e18-9730eb836e44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:23.504640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:23.504640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:23.504640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:23.504640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:23.504640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:23.504640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:23.504640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:23.504640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:23.504640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:23.504640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:23.504640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:23.504640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:23.504640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:23.504640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:23.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:23.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:23.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:23.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:23.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:23.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:23.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:23.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:23.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:23.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:23.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:23.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:23.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:23.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:23.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:23.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:23.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:23.506847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:23.506847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:23.506847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:23.506847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:23.508797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 403/787] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:23.514436 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7366ce3-f665-4daf-8e18-9730eb836e44 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:23.521157 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5c159-afbb-4213-8f12-eff1a3e61a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:23.522479 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5c159-afbb-4213-8f12-eff1a3e61a26 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:23.522961 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5c159-afbb-4213-8f12-eff1a3e61a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:23.523243 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5c159-afbb-4213-8f12-eff1a3e61a26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:23.534540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5c159-afbb-4213-8f12-eff1a3e61a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:23.550450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b9c5c159-afbb-4213-8f12-eff1a3e61a26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:46:23.550450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:23.550450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:23.550450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:23.550450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:23.550450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:23.550450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:23.550450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:23.550450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:23.550450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:23.550450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:23.550450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:23.550450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:23.550450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:23.550450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:23.550450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:23.551488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:23.551488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:23.551488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:23.551488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:23.551488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:23.551488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:23.551488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:23.551488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:23.551488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:23.551488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:23.551488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:23.551488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:23.551488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:23.551488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:23.551488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:23.551488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:23.553065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:23.553065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:23.553065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:23.553065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:23.553065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:23.553065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:23.553065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:23.553065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:23.553065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:23.553065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:23.553065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:46:23.553065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:23.553065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 385/788] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:46:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:23.559677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5c159-afbb-4213-8f12-eff1a3e61a26 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:23.672240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5c159-afbb-4213-8f12-eff1a3e61a26 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:23.708548 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271958b-e816-485d-ab68-1407c44af6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:23.710127 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271958b-e816-485d-ab68-1407c44af6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:23.710423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271958b-e816-485d-ab68-1407c44af6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:23.710736 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271958b-e816-485d-ab68-1407c44af6f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:23.711492 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271958b-e816-485d-ab68-1407c44af6f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:23.711802 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271958b-e816-485d-ab68-1407c44af6f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:23.713288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271958b-e816-485d-ab68-1407c44af6f6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:23.722770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 386/789] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:46:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:23.729776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271958b-e816-485d-ab68-1407c44af6f6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:23.741783 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7366ce3-f665-4daf-8e18-9730eb836e44 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:23.778932 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:23.815287 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b51e0-6fa4-44ad-964f-a5f587488881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:23.815954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b51e0-6fa4-44ad-964f-a5f587488881 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:23.816242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b51e0-6fa4-44ad-964f-a5f587488881 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:23.816519 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b51e0-6fa4-44ad-964f-a5f587488881 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:23.817241 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b51e0-6fa4-44ad-964f-a5f587488881 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:23.817548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b51e0-6fa4-44ad-964f-a5f587488881 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:23.819477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b51e0-6fa4-44ad-964f-a5f587488881 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:23.846746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 404/790] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:23.852630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b51e0-6fa4-44ad-964f-a5f587488881 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:23.900174 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:23.924714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271958b-e816-485d-ab68-1407c44af6f6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:23.990890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b51e0-6fa4-44ad-964f-a5f587488881 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:23.995302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6ecb2-c246-4420-9be2-c8493a0c132b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:23.996118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6ecb2-c246-4420-9be2-c8493a0c132b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:23.996403 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6ecb2-c246-4420-9be2-c8493a0c132b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:23.996683 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6ecb2-c246-4420-9be2-c8493a0c132b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:24.016777 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6ecb2-c246-4420-9be2-c8493a0c132b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:24.018502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6ecb2-c246-4420-9be2-c8493a0c132b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:24.047420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-6eb6ecb2-c246-4420-9be2-c8493a0c132b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:24.047420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:24.047420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:24.047420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:24.047420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:24.047420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:24.047420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:24.047420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:24.047420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:24.047420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:24.047420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:24.047420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:24.047420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:24.047420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:24.047420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:24.048388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:24.048388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:24.048388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:24.048388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:24.048388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:24.048388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:24.048388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:24.048388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:24.048388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:24.048388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:24.048388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:24.048388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:24.048388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:24.048388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:24.048388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:24.048388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:24.048388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:24.049310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:24.049310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:24.049310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:24.049310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:24.049310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 387/791] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:24.056937 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6ecb2-c246-4420-9be2-c8493a0c132b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:24.057739 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c2976-01a9-4e11-90ca-958b23668b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:24.058829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c2976-01a9-4e11-90ca-958b23668b05 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:24.059362 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c2976-01a9-4e11-90ca-958b23668b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:24.059854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c2976-01a9-4e11-90ca-958b23668b05 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:24.083712 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c2976-01a9-4e11-90ca-958b23668b05 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:24.086168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c2976-01a9-4e11-90ca-958b23668b05 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:24.106056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-bb4c2976-01a9-4e11-90ca-958b23668b05 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:46:24.106056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:24.106056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:24.106056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:24.106056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:24.106056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:24.106056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:24.106056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:24.106056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:24.106056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:24.106056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:24.106056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:24.106056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:24.106056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:24.106056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:24.106056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:24.107129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:24.108091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:24.108091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:24.108091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:24.108091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:24.108091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:24.108091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:24.108091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:24.108091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:24.108091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:46:24.108091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:24.122116 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 405/792] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Jun 16 18:46:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:24.127290 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c2976-01a9-4e11-90ca-958b23668b05 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:24.193290 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:24.243904 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6ecb2-c246-4420-9be2-c8493a0c132b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:24.295012 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:24.314121 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c2976-01a9-4e11-90ca-958b23668b05 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:24.327526 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1d2f2-87d0-4ea5-abcf-f08847258859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:24.328558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1d2f2-87d0-4ea5-abcf-f08847258859 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:24.329046 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1d2f2-87d0-4ea5-abcf-f08847258859 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:24.329548 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1d2f2-87d0-4ea5-abcf-f08847258859 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:24.330799 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1d2f2-87d0-4ea5-abcf-f08847258859 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:24.331308 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1d2f2-87d0-4ea5-abcf-f08847258859 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:24.333739 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1d2f2-87d0-4ea5-abcf-f08847258859 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:24.352066 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 388/793] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:24.358006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a0fb1-1e55-4499-af04-c7a9d77e9f13 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:24.358006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a0fb1-1e55-4499-af04-c7a9d77e9f13 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:24.358006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a0fb1-1e55-4499-af04-c7a9d77e9f13 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:24.359406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 389/794] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:24.360117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a0fb1-1e55-4499-af04-c7a9d77e9f13 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:24.368130 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c7eae-908a-4b8d-be28-fa396c6542a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:24.369113 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c7eae-908a-4b8d-be28-fa396c6542a8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:24.369605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c7eae-908a-4b8d-be28-fa396c6542a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:24.370079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c7eae-908a-4b8d-be28-fa396c6542a8 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:24.371297 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c7eae-908a-4b8d-be28-fa396c6542a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:24.371822 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c7eae-908a-4b8d-be28-fa396c6542a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:24.373464 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c7eae-908a-4b8d-be28-fa396c6542a8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:24.393451 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 406/795] 10.222.0.70 () {56 vars in 1149 bytes} [Thu Jun 16 18:46:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:24.395268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e89f02-429e-425b-954d-6b1919a0d768 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:24.395769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e89f02-429e-425b-954d-6b1919a0d768 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:24.396263 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e89f02-429e-425b-954d-6b1919a0d768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:24.414520 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e4e89f02-429e-425b-954d-6b1919a0d768 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:24.416061 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e4e89f02-429e-425b-954d-6b1919a0d768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:24.470871 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a0fb1-1e55-4499-af04-c7a9d77e9f13 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:24.498825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 407/796] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:24.500079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e89f02-429e-425b-954d-6b1919a0d768 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:24.523381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648785cb-31d7-455c-928e-07e616de36ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:24.524414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648785cb-31d7-455c-928e-07e616de36ea None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:24.524596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648785cb-31d7-455c-928e-07e616de36ea None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:24.524847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648785cb-31d7-455c-928e-07e616de36ea None admin] PATH_INFO: `/v3/users/swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:24.541558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648785cb-31d7-455c-928e-07e616de36ea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:24.543088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648785cb-31d7-455c-928e-07e616de36ea None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:24.558744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-648785cb-31d7-455c-928e-07e616de36ea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:46:24.558744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:24.558744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:24.558744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:24.558744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:24.558744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:24.558744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:24.558744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:24.558744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:24.558744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:24.558744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:24.558744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:24.558744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:24.558744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:24.558744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:24.558744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:24.559755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:24.560695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:24.560695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:24.560695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:24.560695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:24.560695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:24.560695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:24.560695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:24.560695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:24.560695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:46:24.560695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:24.562891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:24.565757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 390/797] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:46:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:24.566755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648785cb-31d7-455c-928e-07e616de36ea None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:24.599492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:24.629579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e89f02-429e-425b-954d-6b1919a0d768 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:24.668404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d04995-4759-46ac-b585-7499c0be1e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:24.669339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d04995-4759-46ac-b585-7499c0be1e47 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:24.669686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d04995-4759-46ac-b585-7499c0be1e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:24.670030 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d04995-4759-46ac-b585-7499c0be1e47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:24.681650 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d04995-4759-46ac-b585-7499c0be1e47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:24.683143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d04995-4759-46ac-b585-7499c0be1e47 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:24.685145 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648785cb-31d7-455c-928e-07e616de36ea None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:24.701492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-a5d04995-4759-46ac-b585-7499c0be1e47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:24.701492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:24.701492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:24.701492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:24.701492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:24.701492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:24.701492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:24.701492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:24.701492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:24.701492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:24.701492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:24.701492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:24.701492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:24.701492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:24.701492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:24.702431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:24.702431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:24.702431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:24.702431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:24.702431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:24.702431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:24.702431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:24.702431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:24.702431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:24.702431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:24.702431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:24.702431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:24.702431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:24.702431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:24.702431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:24.702431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:24.702431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:24.703392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:24.703392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:24.703392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:24.703392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:24.706726 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 408/798] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:24.708058 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d04995-4759-46ac-b585-7499c0be1e47 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:24.727885 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe1499-3cda-4f19-b700-6e46ccbc512b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:24.728826 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe1499-3cda-4f19-b700-6e46ccbc512b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:24.729108 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe1499-3cda-4f19-b700-6e46ccbc512b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:24.729527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe1499-3cda-4f19-b700-6e46ccbc512b None admin] PATH_INFO: `/v3/services/network` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:24.730175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe1499-3cda-4f19-b700-6e46ccbc512b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:24.731752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe1499-3cda-4f19-b700-6e46ccbc512b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:24.744574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-21fe1499-3cda-4f19-b700-6e46ccbc512b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:46:24.744574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:24.744574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:24.744574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:24.744574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:24.744574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:24.744574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:24.744574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:24.744574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:46:24.744574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:24.744574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:24.744574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:24.744574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:24.744574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:24.744574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:24.744574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:24.744574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:24.745613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:24.745613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:24.745613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:24.745613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:24.745613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:24.745613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:24.745613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:24.745613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:24.745613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:24.745613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:24.745613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:24.745613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:24.745613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:24.745613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:24.745613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:24.745613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:24.746445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:24.746445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:24.746445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:24.746445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:24.746445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:24.746445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:24.746445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:24.746445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:24.746445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:24.746445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:24.746445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:24.746445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:24.746445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:46:24.746445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:24.746445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 391/799] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:24 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:24.747297 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80552df3-6edf-4598-8b9a-cded468af1de None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:24.747369 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80552df3-6edf-4598-8b9a-cded468af1de None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:24.747981 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80552df3-6edf-4598-8b9a-cded468af1de None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:24.749110 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 392/800] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:24.749810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80552df3-6edf-4598-8b9a-cded468af1de None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:24.818088 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d04995-4759-46ac-b585-7499c0be1e47 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:24.818780 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:24.846279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:24.856385 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019b1b9-798b-40d4-853f-c41f7bb61dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:24.857048 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019b1b9-798b-40d4-853f-c41f7bb61dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:24.857330 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019b1b9-798b-40d4-853f-c41f7bb61dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:24.857602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019b1b9-798b-40d4-853f-c41f7bb61dc4 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:24.858424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019b1b9-798b-40d4-853f-c41f7bb61dc4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:24.858761 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019b1b9-798b-40d4-853f-c41f7bb61dc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:24.860334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019b1b9-798b-40d4-853f-c41f7bb61dc4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:24.864108 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80552df3-6edf-4598-8b9a-cded468af1de None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:24.883458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 409/801] 10.222.0.70 () {56 vars in 1135 bytes} [Thu Jun 16 18:46:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:24.883458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019b1b9-798b-40d4-853f-c41f7bb61dc4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:24.906178 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ccde7-39d5-468d-9240-c63f9b813705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:24.907124 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ccde7-39d5-468d-9240-c63f9b813705 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:24.907409 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ccde7-39d5-468d-9240-c63f9b813705 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:24.907840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ccde7-39d5-468d-9240-c63f9b813705 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:24.908874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ccde7-39d5-468d-9240-c63f9b813705 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:24.909222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ccde7-39d5-468d-9240-c63f9b813705 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:24.910551 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ccde7-39d5-468d-9240-c63f9b813705 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:24.927418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 393/802] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:24.927418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec6a5a-f182-44e4-8c36-73548c61c9ec None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:24.927418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec6a5a-f182-44e4-8c36-73548c61c9ec None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:24.927418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec6a5a-f182-44e4-8c36-73548c61c9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:24.943300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a5ec6a5a-f182-44e4-8c36-73548c61c9ec None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:24.945029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a5ec6a5a-f182-44e4-8c36-73548c61c9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:25.002964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019b1b9-798b-40d4-853f-c41f7bb61dc4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:25.016986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 394/803] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:25.027887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec6a5a-f182-44e4-8c36-73548c61c9ec None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:25.039285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c6377-71b1-4437-8935-a0ce791901a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:25.039999 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c6377-71b1-4437-8935-a0ce791901a1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:25.040275 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c6377-71b1-4437-8935-a0ce791901a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:25.040547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c6377-71b1-4437-8935-a0ce791901a1 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:25.041508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c6377-71b1-4437-8935-a0ce791901a1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:25.041628 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c6377-71b1-4437-8935-a0ce791901a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:25.043003 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c6377-71b1-4437-8935-a0ce791901a1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:25.047976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 410/804] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Jun 16 18:46:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:25.048750 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c6377-71b1-4437-8935-a0ce791901a1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:25.092013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:25.101284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:25.148320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec6a5a-f182-44e4-8c36-73548c61c9ec None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:25.157728 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c6377-71b1-4437-8935-a0ce791901a1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:25.184774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369e79d-dab6-48d3-b737-c835825aa739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:25.185536 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369e79d-dab6-48d3-b737-c835825aa739 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:25.185817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369e79d-dab6-48d3-b737-c835825aa739 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:25.186090 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369e79d-dab6-48d3-b737-c835825aa739 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:25.196399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65698b68-119d-468c-ba04-68f7c571f7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:25.197067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369e79d-dab6-48d3-b737-c835825aa739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:25.197242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65698b68-119d-468c-ba04-68f7c571f7fc None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:25.197489 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65698b68-119d-468c-ba04-68f7c571f7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:25.197768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65698b68-119d-468c-ba04-68f7c571f7fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:25.198517 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369e79d-dab6-48d3-b737-c835825aa739 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:25.204788 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65698b68-119d-468c-ba04-68f7c571f7fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:25.206218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65698b68-119d-468c-ba04-68f7c571f7fc None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:25.216229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c369e79d-dab6-48d3-b737-c835825aa739 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:25.216229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:25.216229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:25.216229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:25.216229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:25.216229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:25.216229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:25.216229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:25.216229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:25.216229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:25.216229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:25.216229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:25.216229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:25.216229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:25.216229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:25.217156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:25.217156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:25.217156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:25.217156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:25.217156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:25.217156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:25.217156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:25.217156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:25.217156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:25.217156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:25.217156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:25.217156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:25.217156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:25.217156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:25.217156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:25.217156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:25.217156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:25.223159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:25.223159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:25.223159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:25.223159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:25.223159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 395/805] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:25.223159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369e79d-dab6-48d3-b737-c835825aa739 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:25.223528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-65698b68-119d-468c-ba04-68f7c571f7fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:25.223528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:25.223528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:25.223528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:25.223528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:25.223528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:25.223528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:25.223528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:25.223528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:25.223528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:25.223528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:25.223528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:25.223528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:25.223528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:25.223528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:25.224338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:25.224338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:25.224338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:25.224338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:25.224338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:25.224338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:25.224338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:25.224338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:25.224338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:25.224338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:25.224338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:25.224338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:25.224338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:25.224338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:25.224338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:25.224338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:25.224338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:25.226460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:25.226460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:25.226460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:25.226460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:25.226460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:25.226460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:25.226460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:25.226460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:25.226460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 411/806] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:25.226460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65698b68-119d-468c-ba04-68f7c571f7fc None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:25.335546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369e79d-dab6-48d3-b737-c835825aa739 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:25.337017 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:25.337540 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65698b68-119d-468c-ba04-68f7c571f7fc None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:25.348084 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:25.375055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be65b1-98b5-45cb-86fd-371d76d557f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:25.375510 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68de53f-d1aa-4f4a-a54b-7a902f23f205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:25.376172 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68de53f-d1aa-4f4a-a54b-7a902f23f205 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:25.376454 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68de53f-d1aa-4f4a-a54b-7a902f23f205 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:25.376735 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68de53f-d1aa-4f4a-a54b-7a902f23f205 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:25.376926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be65b1-98b5-45cb-86fd-371d76d557f0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:25.377388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be65b1-98b5-45cb-86fd-371d76d557f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:25.377500 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68de53f-d1aa-4f4a-a54b-7a902f23f205 None admin] RBAC: Adding query filter params (type=network) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:25.377723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68de53f-d1aa-4f4a-a54b-7a902f23f205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:25.378116 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be65b1-98b5-45cb-86fd-371d76d557f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:25.379111 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68de53f-d1aa-4f4a-a54b-7a902f23f205 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:25.385165 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 412/807] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Jun 16 18:46:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:25.385950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68de53f-d1aa-4f4a-a54b-7a902f23f205 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:25.394200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be65b1-98b5-45cb-86fd-371d76d557f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:25.402484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-16be65b1-98b5-45cb-86fd-371d76d557f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:46:25.402484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:25.402484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:25.402484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:25.402484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:25.402484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:25.402484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:25.402484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:25.402484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:25.402484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:25.402484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:25.402484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:25.402484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:25.402484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:25.402484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:25.402484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:25.403453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:25.403453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:25.403453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:25.403453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:25.403453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:25.403453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:25.403453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:25.403453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:25.403453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:25.403453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:25.403453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:25.403453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:25.403453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:25.403453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:25.403453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:25.403453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:25.404296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:25.404296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:25.404296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:25.404296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:25.404296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:25.404296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:25.404296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:25.404296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:25.404296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:25.404296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:25.404296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:46:25.404296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:25.406421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 396/808] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:46:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:25.407388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be65b1-98b5-45cb-86fd-371d76d557f0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:25.501852 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68de53f-d1aa-4f4a-a54b-7a902f23f205 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:25.517072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be65b1-98b5-45cb-86fd-371d76d557f0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:25.542546 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830fcbd-e2eb-42b6-8c39-d24992401b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:25.543230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830fcbd-e2eb-42b6-8c39-d24992401b78 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:25.543512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830fcbd-e2eb-42b6-8c39-d24992401b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:25.543786 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830fcbd-e2eb-42b6-8c39-d24992401b78 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:25.544506 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830fcbd-e2eb-42b6-8c39-d24992401b78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:25.544810 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830fcbd-e2eb-42b6-8c39-d24992401b78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:25.546165 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830fcbd-e2eb-42b6-8c39-d24992401b78 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:25.554662 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54262861-bacd-4051-99e8-0c0cb6c4ca8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:25.555331 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54262861-bacd-4051-99e8-0c0cb6c4ca8a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:25.555642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54262861-bacd-4051-99e8-0c0cb6c4ca8a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:25.555925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54262861-bacd-4051-99e8-0c0cb6c4ca8a None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:25.556782 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54262861-bacd-4051-99e8-0c0cb6c4ca8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:25.557092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54262861-bacd-4051-99e8-0c0cb6c4ca8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:25.558603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54262861-bacd-4051-99e8-0c0cb6c4ca8a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:25.563653 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 413/809] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:25.563653 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830fcbd-e2eb-42b6-8c39-d24992401b78 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:25.579894 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 397/810] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:25.579894 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54262861-bacd-4051-99e8-0c0cb6c4ca8a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:25.598473 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:25.603705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:25.675206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830fcbd-e2eb-42b6-8c39-d24992401b78 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:25.686594 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54262861-bacd-4051-99e8-0c0cb6c4ca8a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:25.712117 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792899f5-aac9-4640-8777-51b8d92aa51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:25.712778 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792899f5-aac9-4640-8777-51b8d92aa51e None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:25.713063 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792899f5-aac9-4640-8777-51b8d92aa51e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:25.713344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792899f5-aac9-4640-8777-51b8d92aa51e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:25.714075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792899f5-aac9-4640-8777-51b8d92aa51e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:25.715474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792899f5-aac9-4640-8777-51b8d92aa51e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:25.721219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a48ed1-82d1-4a44-94b3-7509b01a7e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:25.721876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a48ed1-82d1-4a44-94b3-7509b01a7e7a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:25.722154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a48ed1-82d1-4a44-94b3-7509b01a7e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:25.722425 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a48ed1-82d1-4a44-94b3-7509b01a7e7a None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:25.723194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a48ed1-82d1-4a44-94b3-7509b01a7e7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:25.723497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a48ed1-82d1-4a44-94b3-7509b01a7e7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:25.725516 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a48ed1-82d1-4a44-94b3-7509b01a7e7a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:25.734986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 398/811] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:46:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:25.735682 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a48ed1-82d1-4a44-94b3-7509b01a7e7a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:25.750893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 414/812] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Jun 16 18:46:25 2022] POST /identity/v3/endpoints => generated 327 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:46:25.750893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792899f5-aac9-4640-8777-51b8d92aa51e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:25.840850 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:25.842503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:25.863099 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a48ed1-82d1-4a44-94b3-7509b01a7e7a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:25.864637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792899f5-aac9-4640-8777-51b8d92aa51e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:25.902457 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf5871-0985-4b67-be91-9662b41daf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:25.903255 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad994d-d859-4f2e-8159-23ebb0dd5c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:25.903374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf5871-0985-4b67-be91-9662b41daf8d None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:25.903634 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf5871-0985-4b67-be91-9662b41daf8d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:25.903906 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf5871-0985-4b67-be91-9662b41daf8d None admin] PATH_INFO: `/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/02c826fc0baa480393df4c884c9e35dc/roles/c123034c11cc4d9ea6cab00e05bdcd53` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:25.904086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad994d-d859-4f2e-8159-23ebb0dd5c36 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:25.904340 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad994d-d859-4f2e-8159-23ebb0dd5c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:25.904617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad994d-d859-4f2e-8159-23ebb0dd5c36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:25.913968 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad994d-d859-4f2e-8159-23ebb0dd5c36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:25.915460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad994d-d859-4f2e-8159-23ebb0dd5c36 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:25.921534 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf5871-0985-4b67-be91-9662b41daf8d None admin] RBAC: Authorizing `identity:create_grant(project_id=34185125e75d4aad8ad5382ab22ee8a2, user_id=02c826fc0baa480393df4c884c9e35dc, role_id=c123034c11cc4d9ea6cab00e05bdcd53)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:25.922976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf5871-0985-4b67-be91-9662b41daf8d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:25.933055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-bbad994d-d859-4f2e-8159-23ebb0dd5c36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:25.933055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:25.933055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:25.933055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:25.933055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:25.933055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:25.933055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:25.933055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:25.933055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:25.933055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:25.933055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:25.933055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:25.933055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:25.933055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:25.933055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:25.934186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:25.934186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:25.934186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:25.934186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:25.934186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:25.934186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:25.934186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:25.934186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:25.934186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:25.934186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:25.934186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:25.934186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:25.934186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:25.934186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:25.934186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:25.934186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:25.934186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:25.935183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:25.935183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:25.935183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:25.935183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:25.935183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:25.935183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:25.935183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:25.935183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:25.935183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 399/813] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:25.936200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874fe19c-506c-4ec8-880c-c2f40b9880b2 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:25.936469 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874fe19c-506c-4ec8-880c-c2f40b9880b2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:25.936747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874fe19c-506c-4ec8-880c-c2f40b9880b2 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:25.938787 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 400/814] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:25.939247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874fe19c-506c-4ec8-880c-c2f40b9880b2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:25.946204 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 415/815] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:46:25 2022] PUT /identity/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/02c826fc0baa480393df4c884c9e35dc/roles/c123034c11cc4d9ea6cab00e05bdcd53 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:46:25.946204 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf5871-0985-4b67-be91-9662b41daf8d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:26.075625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf5871-0985-4b67-be91-9662b41daf8d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:26.081177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874fe19c-506c-4ec8-880c-c2f40b9880b2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:26.113893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4f4ce-b31e-49ff-93c7-9d6fc58170f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:26.114438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4f4ce-b31e-49ff-93c7-9d6fc58170f6 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:26.114749 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4f4ce-b31e-49ff-93c7-9d6fc58170f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:26.115020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4f4ce-b31e-49ff-93c7-9d6fc58170f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:26.115971 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4f4ce-b31e-49ff-93c7-9d6fc58170f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:26.116279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4f4ce-b31e-49ff-93c7-9d6fc58170f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:26.118625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4f4ce-b31e-49ff-93c7-9d6fc58170f6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:26.120126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:26.125292 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4647bd1-8d8c-4dd0-ae55-27a458e3ff48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:26.126323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4647bd1-8d8c-4dd0-ae55-27a458e3ff48 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:26.126779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4647bd1-8d8c-4dd0-ae55-27a458e3ff48 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:26.127703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4647bd1-8d8c-4dd0-ae55-27a458e3ff48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:26.135832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 416/816] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:26.137464 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f267a7-b06d-47f8-85f0-e5e136a79aae None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:26.137845 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f267a7-b06d-47f8-85f0-e5e136a79aae None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:26.138215 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f267a7-b06d-47f8-85f0-e5e136a79aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:26.142153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4647bd1-8d8c-4dd0-ae55-27a458e3ff48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:26.144500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4647bd1-8d8c-4dd0-ae55-27a458e3ff48 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:26.150721 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:26.158313 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-59f267a7-b06d-47f8-85f0-e5e136a79aae None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:26.159648 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-59f267a7-b06d-47f8-85f0-e5e136a79aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:26.167363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c4647bd1-8d8c-4dd0-ae55-27a458e3ff48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:26.167363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:26.167363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:26.167363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:26.167363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:26.167363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:26.167363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:26.167363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:26.167363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:26.167363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:26.167363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:26.167363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:26.167363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:26.167363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:26.167363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:26.168392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:26.168392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:26.168392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:26.168392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:26.168392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:26.168392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:26.168392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:26.168392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:26.168392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:26.168392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:26.168392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:26.168392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:26.168392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:26.168392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:26.168392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:26.168392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:26.168392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:26.169447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:26.169447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:26.169447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:26.169447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:26.169447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 401/817] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:26.171366 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4647bd1-8d8c-4dd0-ae55-27a458e3ff48 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:26.233861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 417/818] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:26.234753 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f267a7-b06d-47f8-85f0-e5e136a79aae None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:26.289139 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4647bd1-8d8c-4dd0-ae55-27a458e3ff48 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:26.324556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905d5c8-bfb6-48c9-8615-240b97d728dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:26.325617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905d5c8-bfb6-48c9-8615-240b97d728dd None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:26.325997 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905d5c8-bfb6-48c9-8615-240b97d728dd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:26.326375 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905d5c8-bfb6-48c9-8615-240b97d728dd None admin] PATH_INFO: `/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/7b22fe0ab0354385924923d0d443acae/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:26.328508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905d5c8-bfb6-48c9-8615-240b97d728dd None admin] RBAC: Authorizing `identity:create_grant(project_id=34185125e75d4aad8ad5382ab22ee8a2, user_id=7b22fe0ab0354385924923d0d443acae, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:26.330064 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905d5c8-bfb6-48c9-8615-240b97d728dd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:26.344114 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f267a7-b06d-47f8-85f0-e5e136a79aae None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:26.355687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 402/819] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:46:26 2022] PUT /identity/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/7b22fe0ab0354385924923d0d443acae/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:46:26.355687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905d5c8-bfb6-48c9-8615-240b97d728dd None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:26.387823 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:26.403223 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af167a-fa32-45f5-b26f-c978464cedb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:26.404090 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af167a-fa32-45f5-b26f-c978464cedb2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:26.404862 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af167a-fa32-45f5-b26f-c978464cedb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:26.405220 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af167a-fa32-45f5-b26f-c978464cedb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:26.406323 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af167a-fa32-45f5-b26f-c978464cedb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:26.406739 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af167a-fa32-45f5-b26f-c978464cedb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:26.409116 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af167a-fa32-45f5-b26f-c978464cedb2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:26.422044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:26.429087 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 418/820] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:26.434358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af167a-fa32-45f5-b26f-c978464cedb2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:26.499329 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905d5c8-bfb6-48c9-8615-240b97d728dd None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:26.547214 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c0676-6998-45b8-a74d-57389a332b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:26.548145 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c0676-6998-45b8-a74d-57389a332b60 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:26.548526 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c0676-6998-45b8-a74d-57389a332b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:26.548906 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c0676-6998-45b8-a74d-57389a332b60 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:26.549675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c0676-6998-45b8-a74d-57389a332b60 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:26.551191 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c0676-6998-45b8-a74d-57389a332b60 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:26.565176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c50c0676-6998-45b8-a74d-57389a332b60 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:46:26.565176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:26.565176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:26.565176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:26.565176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:26.565176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:26.565176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:26.565176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:26.565176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:46:26.565176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:26.565176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:26.565176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:26.565176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:26.565176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:26.565176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:26.565176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:26.566796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:26.566796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:26.566796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:26.566796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:26.566796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:26.566796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:26.566796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:26.566796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:26.566796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:26.566796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:26.566796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:26.566796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:26.566796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:26.566796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:26.566796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:26.566796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:26.567926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:26.567926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:26.567926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:26.567926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:26.567926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:26.567926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:26.567926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:26.567926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:26.567926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:26.567926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:26.567926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:26.567926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:26.567926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:26.567926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:46:26.567926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:26.569510 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 403/821] 10.222.0.70 () {56 vars in 1129 bytes} [Thu Jun 16 18:46:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:26.575473 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c0676-6998-45b8-a74d-57389a332b60 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:26.576563 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af167a-fa32-45f5-b26f-c978464cedb2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:26.617922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ee3fa-886c-4248-841c-7bb55a3b50aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:26.619999 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ee3fa-886c-4248-841c-7bb55a3b50aa None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:26.620567 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ee3fa-886c-4248-841c-7bb55a3b50aa None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:26.621007 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ee3fa-886c-4248-841c-7bb55a3b50aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:26.632361 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ee3fa-886c-4248-841c-7bb55a3b50aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:26.634659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ee3fa-886c-4248-841c-7bb55a3b50aa None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:26.654605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-c75ee3fa-886c-4248-841c-7bb55a3b50aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:46:26.654605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:26.654605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:26.654605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:26.654605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:26.654605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:26.654605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:26.654605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:26.654605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:26.654605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:26.654605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:26.654605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:26.654605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:26.654605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:26.654605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:26.654605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:26.655791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:26.656965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:26.656965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:26.656965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:26.656965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:26.656965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:26.656965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:26.656965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:26.656965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:26.656965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:46:26.656965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:26.664974 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 419/822] 10.222.0.70 () {56 vars in 1093 bytes} [Thu Jun 16 18:46:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:26.666436 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ee3fa-886c-4248-841c-7bb55a3b50aa None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:26.710512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c0676-6998-45b8-a74d-57389a332b60 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:26.729247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:26.737413 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:26.758233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6ff68-c5f5-4248-b7d0-d3d4c91a0dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:26.759285 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6ff68-c5f5-4248-b7d0-d3d4c91a0dfb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:26.759675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6ff68-c5f5-4248-b7d0-d3d4c91a0dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:26.760058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6ff68-c5f5-4248-b7d0-d3d4c91a0dfb None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:26.760919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c6ff68-c5f5-4248-b7d0-d3d4c91a0dfb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:26.761276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c6ff68-c5f5-4248-b7d0-d3d4c91a0dfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:26.762955 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c6ff68-c5f5-4248-b7d0-d3d4c91a0dfb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:26.774469 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 404/823] 10.222.0.70 () {56 vars in 1124 bytes} [Thu Jun 16 18:46:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:26.775685 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6ff68-c5f5-4248-b7d0-d3d4c91a0dfb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:26.817259 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ee3fa-886c-4248-841c-7bb55a3b50aa None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:26.864682 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9f82e-2ec4-4235-9dee-85f695d61dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:26.865891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9f82e-2ec4-4235-9dee-85f695d61dbf None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:26.866324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9f82e-2ec4-4235-9dee-85f695d61dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:26.866809 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9f82e-2ec4-4235-9dee-85f695d61dbf None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:26.868070 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9f82e-2ec4-4235-9dee-85f695d61dbf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:26.869020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9f82e-2ec4-4235-9dee-85f695d61dbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:26.870889 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9f82e-2ec4-4235-9dee-85f695d61dbf None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:26.889421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6ff68-c5f5-4248-b7d0-d3d4c91a0dfb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:26.901656 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 420/824] 10.222.0.70 () {56 vars in 1133 bytes} [Thu Jun 16 18:46:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:26.903363 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9f82e-2ec4-4235-9dee-85f695d61dbf None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:26.932652 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbeaa44-b504-4e81-b6e7-c2531fe40d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:26.933393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbeaa44-b504-4e81-b6e7-c2531fe40d40 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:26.933678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbeaa44-b504-4e81-b6e7-c2531fe40d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:26.933953 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbeaa44-b504-4e81-b6e7-c2531fe40d40 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:26.934806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbeaa44-b504-4e81-b6e7-c2531fe40d40 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:26.935057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbeaa44-b504-4e81-b6e7-c2531fe40d40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:26.936415 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbeaa44-b504-4e81-b6e7-c2531fe40d40 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:26.946751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 405/825] 10.222.0.70 () {56 vars in 1124 bytes} [Thu Jun 16 18:46:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:26.948311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbeaa44-b504-4e81-b6e7-c2531fe40d40 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:27.004889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:27.024893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9f82e-2ec4-4235-9dee-85f695d61dbf None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:27.026905 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:27.063364 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbeaa44-b504-4e81-b6e7-c2531fe40d40 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:27.071230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c999dc-edc0-4a1c-bdf8-b36b0af4388a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:27.071918 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c999dc-edc0-4a1c-bdf8-b36b0af4388a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:27.072198 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c999dc-edc0-4a1c-bdf8-b36b0af4388a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:27.072473 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c999dc-edc0-4a1c-bdf8-b36b0af4388a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:27.108772 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae393887-5048-44a7-bed5-acacfc19f734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:27.109867 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae393887-5048-44a7-bed5-acacfc19f734 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:27.110358 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae393887-5048-44a7-bed5-acacfc19f734 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:27.110860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae393887-5048-44a7-bed5-acacfc19f734 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:27.111811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae393887-5048-44a7-bed5-acacfc19f734 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:27.113488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae393887-5048-44a7-bed5-acacfc19f734 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:27.140417 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 406/826] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Jun 16 18:46:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:46:27.159678 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c999dc-edc0-4a1c-bdf8-b36b0af4388a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:27.161239 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c999dc-edc0-4a1c-bdf8-b36b0af4388a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:27.282666 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 421/827] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:46:26 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:46:27.289314 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae393887-5048-44a7-bed5-acacfc19f734 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:27.402860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:27.407033 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae393887-5048-44a7-bed5-acacfc19f734 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:27.442796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5c349-99c0-4610-889a-17fec7e73b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:27.444560 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5c349-99c0-4610-889a-17fec7e73b65 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:27.445052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5c349-99c0-4610-889a-17fec7e73b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:27.445555 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5c349-99c0-4610-889a-17fec7e73b65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:27.456757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5c349-99c0-4610-889a-17fec7e73b65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:27.458451 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5c349-99c0-4610-889a-17fec7e73b65 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:27.477664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-34e5c349-99c0-4610-889a-17fec7e73b65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:27.477664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:27.477664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:27.477664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:27.477664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:27.477664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:27.477664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:27.477664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:27.477664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:27.477664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:27.477664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:27.477664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:27.477664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:27.477664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:27.477664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:27.478677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:27.478677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:27.478677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:27.478677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:27.478677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:27.478677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:27.478677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:27.478677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:27.478677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:27.478677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:27.478677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:27.478677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:27.478677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:27.478677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:27.478677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:27.478677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:27.478677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:27.479747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:27.479747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:27.479747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:27.479747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:27.484905 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 407/828] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:27.487510 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c999dc-edc0-4a1c-bdf8-b36b0af4388a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:27.490344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:27.605495 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c999dc-edc0-4a1c-bdf8-b36b0af4388a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:27.642057 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c26a14-185c-4545-a7b0-0ab529c55dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:27.643340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c26a14-185c-4545-a7b0-0ab529c55dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:27.643802 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c26a14-185c-4545-a7b0-0ab529c55dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:27.644298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c26a14-185c-4545-a7b0-0ab529c55dc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:27.645248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c26a14-185c-4545-a7b0-0ab529c55dc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:27.645757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c26a14-185c-4545-a7b0-0ab529c55dc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:27.647997 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c26a14-185c-4545-a7b0-0ab529c55dc3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:27.664889 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 422/829] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:27.666502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5c349-99c0-4610-889a-17fec7e73b65 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:27.775946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5c349-99c0-4610-889a-17fec7e73b65 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:27.810653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9fced-e64d-4e5f-872e-e239f6c6250a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:27.811452 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9fced-e64d-4e5f-872e-e239f6c6250a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:27.811733 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9fced-e64d-4e5f-872e-e239f6c6250a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:27.812006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9fced-e64d-4e5f-872e-e239f6c6250a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:27.819594 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9fced-e64d-4e5f-872e-e239f6c6250a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:27.821042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9fced-e64d-4e5f-872e-e239f6c6250a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:27.835576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ebf9fced-e64d-4e5f-872e-e239f6c6250a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:27.835576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:27.835576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:27.835576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:27.835576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:27.835576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:27.835576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:27.835576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:27.835576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:27.835576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:27.835576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:27.835576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:27.835576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:27.835576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:27.835576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:27.836556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:27.836556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:27.836556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:27.836556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:27.836556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:27.836556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:27.836556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:27.836556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:27.836556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:27.836556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:27.836556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:27.836556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:27.836556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:27.836556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:27.836556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:27.836556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:27.836556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:27.837643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:27.837643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:27.837643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:27.837643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:27.837643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:27.837643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:27.837643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:27.837643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:27.837643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 408/830] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:27.844442 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c26a14-185c-4545-a7b0-0ab529c55dc3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:27.920312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:27.964938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c26a14-185c-4545-a7b0-0ab529c55dc3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:28.000670 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec215bab-3532-423b-8cab-107b80350eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:28.001498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec215bab-3532-423b-8cab-107b80350eff None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:28.001954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec215bab-3532-423b-8cab-107b80350eff None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:28.002335 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec215bab-3532-423b-8cab-107b80350eff None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:28.003346 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec215bab-3532-423b-8cab-107b80350eff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:28.003761 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec215bab-3532-423b-8cab-107b80350eff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:28.005286 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec215bab-3532-423b-8cab-107b80350eff None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:28.024960 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 423/831] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:28.028332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9fced-e64d-4e5f-872e-e239f6c6250a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:28.047413 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:28.147027 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9fced-e64d-4e5f-872e-e239f6c6250a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:28.195113 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f540c-eb10-49d4-9aa7-ec5ccba43e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:28.196072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f540c-eb10-49d4-9aa7-ec5ccba43e62 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:28.196502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f540c-eb10-49d4-9aa7-ec5ccba43e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:28.196921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f540c-eb10-49d4-9aa7-ec5ccba43e62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:28.197925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f540c-eb10-49d4-9aa7-ec5ccba43e62 None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=8342da838fe4462b9de2817ad3ac4ff7) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:28.198386 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f540c-eb10-49d4-9aa7-ec5ccba43e62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:28.199982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f540c-eb10-49d4-9aa7-ec5ccba43e62 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:28.207065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 409/832] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:46:28 2022] GET /identity/v3/role_assignments?user.id=8342da838fe4462b9de2817ad3ac4ff7&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:28.228467 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c514f-bd98-47f1-9b7a-166c26e15ada None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:28.228771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c514f-bd98-47f1-9b7a-166c26e15ada None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:28.229202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c514f-bd98-47f1-9b7a-166c26e15ada None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:28.234867 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 424/833] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:28.240754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652becb8-e07e-48ee-8064-f6877d6289af None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:28.241408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652becb8-e07e-48ee-8064-f6877d6289af None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:28.241866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652becb8-e07e-48ee-8064-f6877d6289af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:28.263595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-652becb8-e07e-48ee-8064-f6877d6289af None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:28.265083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-652becb8-e07e-48ee-8064-f6877d6289af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:28.355270 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 410/834] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:28.380109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c514f-bd98-47f1-9b7a-166c26e15ada None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:28.509008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c514f-bd98-47f1-9b7a-166c26e15ada None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:28.539050 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:28.553883 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2494f76-eb63-415b-a913-222718059300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:28.555126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2494f76-eb63-415b-a913-222718059300 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:28.555603 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2494f76-eb63-415b-a913-222718059300 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:28.556069 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2494f76-eb63-415b-a913-222718059300 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:28.569645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2494f76-eb63-415b-a913-222718059300 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:28.578207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-d2494f76-eb63-415b-a913-222718059300 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:28.578207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:28.578207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:28.578207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:28.578207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:28.578207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:28.578207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:28.578207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:28.578207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:28.578207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:28.578207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:28.578207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:28.578207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:28.578207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:28.578207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:28.578207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:28.579178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:28.579178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:28.579178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:28.579178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:28.579178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:28.579178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:28.579178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:28.579178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:28.579178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:28.579178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:28.579178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:28.579178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:28.579178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:28.579178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:28.579178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:28.579178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:28.580016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:28.580016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:28.580016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:28.580016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:28.580016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:28.580016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:28.580016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:28.580016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:28.580016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:28.580016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:28.580016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:46:28.580016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:28.587028 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 425/835] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:46:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:28.591138 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652becb8-e07e-48ee-8064-f6877d6289af None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:28.600647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:28.718644 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652becb8-e07e-48ee-8064-f6877d6289af None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:28.745663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90e929-8084-45ff-8771-ad45e0982d40 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:28.746054 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90e929-8084-45ff-8771-ad45e0982d40 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:28.746335 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90e929-8084-45ff-8771-ad45e0982d40 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:28.748267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 426/836] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:28.754310 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7722820-3877-4528-b8b0-1a64eb67189e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:28.754784 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7722820-3877-4528-b8b0-1a64eb67189e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:28.754887 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7722820-3877-4528-b8b0-1a64eb67189e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:28.780085 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b7722820-3877-4528-b8b0-1a64eb67189e None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:28.781232 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b7722820-3877-4528-b8b0-1a64eb67189e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:28.882727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 427/837] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:46:28.891610 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5938ec-b012-4694-9b54-e01f1c809756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:28.892985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5938ec-b012-4694-9b54-e01f1c809756 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:28.893399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5938ec-b012-4694-9b54-e01f1c809756 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:28.893677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5938ec-b012-4694-9b54-e01f1c809756 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:28.894435 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5938ec-b012-4694-9b54-e01f1c809756 None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:28.894762 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5938ec-b012-4694-9b54-e01f1c809756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:28.896125 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5938ec-b012-4694-9b54-e01f1c809756 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:28.905940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 411/838] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:46:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:28.911538 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7722820-3877-4528-b8b0-1a64eb67189e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:28.915587 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5938ec-b012-4694-9b54-e01f1c809756 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:29.021633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:29.029309 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7722820-3877-4528-b8b0-1a64eb67189e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:29.045666 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:29.070739 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5938ec-b012-4694-9b54-e01f1c809756 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:29.074206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ffe33-f0e8-42ab-b9d0-4e6b070b2657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:29.075263 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ffe33-f0e8-42ab-b9d0-4e6b070b2657 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:29.075758 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ffe33-f0e8-42ab-b9d0-4e6b070b2657 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:29.076223 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ffe33-f0e8-42ab-b9d0-4e6b070b2657 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:29.088485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ffe33-f0e8-42ab-b9d0-4e6b070b2657 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:29.090324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ffe33-f0e8-42ab-b9d0-4e6b070b2657 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:29.111879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-300ffe33-f0e8-42ab-b9d0-4e6b070b2657 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:29.111879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:29.111879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:29.111879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:29.111879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:29.111879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:29.111879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:29.111879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:29.111879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:29.111879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:29.111879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:29.111879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:29.111879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:29.111879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:29.111879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:29.112846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:29.112846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:29.112846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:29.112846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:29.112846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:29.112846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:29.112846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:29.112846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:29.112846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:29.112846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:29.112846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:29.112846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:29.112846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:29.112846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:29.112846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:29.112846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:29.112846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:29.113738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:29.113738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:29.113738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:29.113738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:29.115551 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07057717-1a41-411e-84fb-5a43b07a5a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:29.116357 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07057717-1a41-411e-84fb-5a43b07a5a9c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:29.116759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07057717-1a41-411e-84fb-5a43b07a5a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:29.117055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07057717-1a41-411e-84fb-5a43b07a5a9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:29.123163 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 428/839] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:29.127296 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ffe33-f0e8-42ab-b9d0-4e6b070b2657 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:29.131515 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07057717-1a41-411e-84fb-5a43b07a5a9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:29.140086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-07057717-1a41-411e-84fb-5a43b07a5a9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:46:29.140086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:29.140086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:29.140086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:29.140086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:29.140086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:29.140086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:29.140086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:29.140086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:29.140086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:29.140086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:29.140086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:29.140086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:29.140086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:29.140086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:29.140086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:29.141062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:29.141062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:29.141062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:29.141062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:29.141062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:29.141062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:29.141062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:29.141062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:29.141062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:29.141062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:29.141062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:29.141062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:29.141062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:29.141062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:29.141062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:29.141062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:29.141911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:29.141911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:29.141911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:29.141911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:29.141911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:29.141911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:29.141911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:29.141911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:29.141911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:29.141911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:29.141911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:46:29.141911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:29.147364 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 412/840] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:46:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:29.151838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07057717-1a41-411e-84fb-5a43b07a5a9c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:29.242770 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ffe33-f0e8-42ab-b9d0-4e6b070b2657 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:29.265305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07057717-1a41-411e-84fb-5a43b07a5a9c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:29.280988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d00a85-d307-468f-8c12-166d5f90e119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:29.281931 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d00a85-d307-468f-8c12-166d5f90e119 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:29.282430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d00a85-d307-468f-8c12-166d5f90e119 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:29.282942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d00a85-d307-468f-8c12-166d5f90e119 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:29.283954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d00a85-d307-468f-8c12-166d5f90e119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:29.284526 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d00a85-d307-468f-8c12-166d5f90e119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:29.286092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d00a85-d307-468f-8c12-166d5f90e119 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:29.293977 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:29.305872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bef88-ea6b-4b33-bb36-9b3fc64585ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:29.305872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bef88-ea6b-4b33-bb36-9b3fc64585ed None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:29.305872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bef88-ea6b-4b33-bb36-9b3fc64585ed None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:29.305872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bef88-ea6b-4b33-bb36-9b3fc64585ed None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:29.306253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297bef88-ea6b-4b33-bb36-9b3fc64585ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:29.306253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297bef88-ea6b-4b33-bb36-9b3fc64585ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:29.306632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297bef88-ea6b-4b33-bb36-9b3fc64585ed None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:29.307073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 429/841] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:29.308240 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d00a85-d307-468f-8c12-166d5f90e119 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:29.321122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 413/842] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:46:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:29.322333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bef88-ea6b-4b33-bb36-9b3fc64585ed None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:29.337409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:29.420346 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d00a85-d307-468f-8c12-166d5f90e119 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:29.436475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bef88-ea6b-4b33-bb36-9b3fc64585ed None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:29.458494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586de32-417d-4a6a-827a-7dac91b094c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:29.460006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586de32-417d-4a6a-827a-7dac91b094c1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:29.460500 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586de32-417d-4a6a-827a-7dac91b094c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:29.460983 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586de32-417d-4a6a-827a-7dac91b094c1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:29.469805 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5586de32-417d-4a6a-827a-7dac91b094c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:29.471545 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5586de32-417d-4a6a-827a-7dac91b094c1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:29.474911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b929c-0179-4c4f-8447-95a279d1f2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:29.475722 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b929c-0179-4c4f-8447-95a279d1f2cf None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:29.476011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b929c-0179-4c4f-8447-95a279d1f2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:29.476290 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b929c-0179-4c4f-8447-95a279d1f2cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:29.487858 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b929c-0179-4c4f-8447-95a279d1f2cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:29.488433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-5586de32-417d-4a6a-827a-7dac91b094c1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:46:29.488433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:29.488433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:29.488433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:29.488433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:29.488433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:29.488433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:29.488433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:29.488433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:29.488433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:29.488433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:29.488433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:29.488433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:29.488433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:29.488433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:29.488433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:29.489355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:29.491503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:29.491503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:29.491503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:29.491503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:29.491503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:29.491503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:29.491503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:29.491503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:29.491503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:46:29.491503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:29.492054 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b929c-0179-4c4f-8447-95a279d1f2cf None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:29.497285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 430/843] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Jun 16 18:46:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:29.499801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586de32-417d-4a6a-827a-7dac91b094c1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:29.512043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-cf7b929c-0179-4c4f-8447-95a279d1f2cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:29.512043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:29.512043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:29.512043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:29.512043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:29.512043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:29.512043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:29.512043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:29.512043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:29.512043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:29.512043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:29.512043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:29.512043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:29.512043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:29.512043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:29.513053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:29.513053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:29.513053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:29.513053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:29.513053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:29.513053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:29.513053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:29.513053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:29.513053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:29.513053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:29.513053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:29.513053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:29.513053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:29.513053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:29.513053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:29.513053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:29.513053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:29.513982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:29.513982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:29.513982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:29.513982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:29.518621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 414/844] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:29.519743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b929c-0179-4c4f-8447-95a279d1f2cf None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:29.569231 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:29.611462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:29.630064 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586de32-417d-4a6a-827a-7dac91b094c1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:29.639921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b929c-0179-4c4f-8447-95a279d1f2cf None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:29.670271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12639e72-f3b9-4fad-a18b-03b3b72a564a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:29.670968 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12639e72-f3b9-4fad-a18b-03b3b72a564a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:29.671247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12639e72-f3b9-4fad-a18b-03b3b72a564a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:29.671525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12639e72-f3b9-4fad-a18b-03b3b72a564a None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:29.672349 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12639e72-f3b9-4fad-a18b-03b3b72a564a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:29.672655 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12639e72-f3b9-4fad-a18b-03b3b72a564a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:29.674062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12639e72-f3b9-4fad-a18b-03b3b72a564a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:29.676732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dac79-c393-45b5-bbe3-6115a395402f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:29.677539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dac79-c393-45b5-bbe3-6115a395402f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:29.677990 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dac79-c393-45b5-bbe3-6115a395402f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:29.678455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dac79-c393-45b5-bbe3-6115a395402f None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:29.679284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937dac79-c393-45b5-bbe3-6115a395402f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:29.679608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937dac79-c393-45b5-bbe3-6115a395402f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:29.681018 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937dac79-c393-45b5-bbe3-6115a395402f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:29.698609 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 415/845] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:29.698609 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01971755-155a-4db3-86a8-82b1a9ba9809 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:29.698609 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01971755-155a-4db3-86a8-82b1a9ba9809 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:29.698609 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01971755-155a-4db3-86a8-82b1a9ba9809 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:29.699450 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 431/846] 10.222.0.70 () {56 vars in 1149 bytes} [Thu Jun 16 18:46:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:29.700514 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12639e72-f3b9-4fad-a18b-03b3b72a564a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:29.701683 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 416/847] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:29.702491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01971755-155a-4db3-86a8-82b1a9ba9809 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:29.808719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12639e72-f3b9-4fad-a18b-03b3b72a564a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:29.816564 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01971755-155a-4db3-86a8-82b1a9ba9809 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:29.819928 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:29.844528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e80155-c65f-4b38-91c2-767225c1010a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:29.845184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e80155-c65f-4b38-91c2-767225c1010a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:29.845477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e80155-c65f-4b38-91c2-767225c1010a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:29.845753 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e80155-c65f-4b38-91c2-767225c1010a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:29.857820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76caa2c9-f8bb-4a8e-8c32-0886254f7da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:29.858595 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:29.859639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76caa2c9-f8bb-4a8e-8c32-0886254f7da8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:29.859986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76caa2c9-f8bb-4a8e-8c32-0886254f7da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:29.860300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76caa2c9-f8bb-4a8e-8c32-0886254f7da8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:29.869312 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76caa2c9-f8bb-4a8e-8c32-0886254f7da8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:29.870957 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76caa2c9-f8bb-4a8e-8c32-0886254f7da8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:29.886634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-76caa2c9-f8bb-4a8e-8c32-0886254f7da8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:46:29.886634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:29.886634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:29.886634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:29.886634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:29.886634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:29.886634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:29.886634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:29.886634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:29.886634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:29.886634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:29.886634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:29.886634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:29.886634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:29.886634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:29.886634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:29.887608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:29.888553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:29.888553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:29.888553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:29.888553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:29.888553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:29.888553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:29.888553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:29.888553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:29.888553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:46:29.888553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:29.888553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 417/848] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:46:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:29.889656 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dd6b3-7226-4bef-ad4c-9dcf7298aca9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:29.889656 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dd6b3-7226-4bef-ad4c-9dcf7298aca9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:29.889656 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dd6b3-7226-4bef-ad4c-9dcf7298aca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:29.908194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-bc9dd6b3-7226-4bef-ad4c-9dcf7298aca9 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:29.909475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bc9dd6b3-7226-4bef-ad4c-9dcf7298aca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:29.925131 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e80155-c65f-4b38-91c2-767225c1010a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:29.926649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e80155-c65f-4b38-91c2-767225c1010a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:29.983612 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 418/849] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:29.984840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9dd6b3-7226-4bef-ad4c-9dcf7298aca9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:30.051788 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 432/850] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:46:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:46:30.051788 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e80155-c65f-4b38-91c2-767225c1010a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:30.087071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:30.097925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9dd6b3-7226-4bef-ad4c-9dcf7298aca9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:30.102091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:30.136996 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab324b-4992-4015-ac3a-19482f2dbe9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:30.137687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab324b-4992-4015-ac3a-19482f2dbe9c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:30.137958 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab324b-4992-4015-ac3a-19482f2dbe9c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:30.138233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab324b-4992-4015-ac3a-19482f2dbe9c None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:30.139569 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab324b-4992-4015-ac3a-19482f2dbe9c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:30.139883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab324b-4992-4015-ac3a-19482f2dbe9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:30.141981 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab324b-4992-4015-ac3a-19482f2dbe9c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:30.162565 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 419/851] 10.222.0.70 () {56 vars in 1135 bytes} [Thu Jun 16 18:46:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:30.163425 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab324b-4992-4015-ac3a-19482f2dbe9c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:30.164952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e80155-c65f-4b38-91c2-767225c1010a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:30.206328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed478d81-0de9-487d-aff7-c786546d746b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:30.207151 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed478d81-0de9-487d-aff7-c786546d746b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:30.207728 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed478d81-0de9-487d-aff7-c786546d746b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:30.207728 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed478d81-0de9-487d-aff7-c786546d746b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:30.208336 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed478d81-0de9-487d-aff7-c786546d746b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:30.210296 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed478d81-0de9-487d-aff7-c786546d746b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:30.226469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ed478d81-0de9-487d-aff7-c786546d746b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:46:30.226469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:30.226469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:30.226469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:30.226469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:30.226469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:30.226469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:30.226469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:30.226469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:46:30.226469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:30.226469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:30.226469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:30.226469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:30.226469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:30.226469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:30.226469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:30.226469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:30.227556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:30.227556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:30.227556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:30.227556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:30.227556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:30.227556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:30.227556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:30.227556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:30.227556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:30.227556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:30.227556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:30.227556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:30.227556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:30.227556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:30.227556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:30.227556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:30.228434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:30.228434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:30.228434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:30.228434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:30.228434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:30.228434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:30.228434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:30.228434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:30.228434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:30.228434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:30.228434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:30.228434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:30.228434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:46:30.228434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:30.228434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 433/852] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:46:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:30.229247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed478d81-0de9-487d-aff7-c786546d746b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:30.275270 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab324b-4992-4015-ac3a-19482f2dbe9c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:30.312043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32addfe1-0ba1-4b9f-abc4-1f32a29d8f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:30.312830 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32addfe1-0ba1-4b9f-abc4-1f32a29d8f5b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:30.313111 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32addfe1-0ba1-4b9f-abc4-1f32a29d8f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:30.313382 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32addfe1-0ba1-4b9f-abc4-1f32a29d8f5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:30.324578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32addfe1-0ba1-4b9f-abc4-1f32a29d8f5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:30.326000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32addfe1-0ba1-4b9f-abc4-1f32a29d8f5b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:30.343684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-32addfe1-0ba1-4b9f-abc4-1f32a29d8f5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:30.343684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:30.343684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:30.343684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:30.343684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:30.343684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:30.343684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:30.343684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:30.343684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:30.343684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:30.343684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:30.343684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:30.343684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:30.343684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:30.343684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:30.344578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:30.344578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:30.344578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:30.344578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:30.344578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:30.344578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:30.344578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:30.344578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:30.344578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:30.344578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:30.344578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:30.344578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:30.344578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:30.344578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:30.344578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:30.344578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:30.344578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:30.345453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:30.345453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:30.345453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:30.345453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:30.345453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 420/853] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:30.345739 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32addfe1-0ba1-4b9f-abc4-1f32a29d8f5b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:30.347420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed478d81-0de9-487d-aff7-c786546d746b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:30.352936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:30.358097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:30.389711 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8792a6b-578e-4793-8c4e-017ea6d67d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:30.390356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8792a6b-578e-4793-8c4e-017ea6d67d2d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:30.390665 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8792a6b-578e-4793-8c4e-017ea6d67d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:30.390940 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8792a6b-578e-4793-8c4e-017ea6d67d2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:30.453915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32addfe1-0ba1-4b9f-abc4-1f32a29d8f5b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:30.463654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8792a6b-578e-4793-8c4e-017ea6d67d2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:30.465388 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8792a6b-578e-4793-8c4e-017ea6d67d2d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:30.491322 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f6272-f9ec-4a4e-a144-8d318a4268fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:30.493032 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f6272-f9ec-4a4e-a144-8d318a4268fe None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:30.493303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f6272-f9ec-4a4e-a144-8d318a4268fe None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:30.493581 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f6272-f9ec-4a4e-a144-8d318a4268fe None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:30.494316 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f6272-f9ec-4a4e-a144-8d318a4268fe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:30.494724 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f6272-f9ec-4a4e-a144-8d318a4268fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:30.496150 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f6272-f9ec-4a4e-a144-8d318a4268fe None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:30.501324 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 421/854] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:46:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:30.502201 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f6272-f9ec-4a4e-a144-8d318a4268fe None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:30.578983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:30.586620 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 434/855] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:46:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:46:30.586620 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8792a6b-578e-4793-8c4e-017ea6d67d2d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:30.596369 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:30.620071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f6272-f9ec-4a4e-a144-8d318a4268fe None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:30.654496 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4e90d-707c-469a-a10c-9a2724508d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:30.655195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4e90d-707c-469a-a10c-9a2724508d7d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:30.655481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4e90d-707c-469a-a10c-9a2724508d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:30.655758 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4e90d-707c-469a-a10c-9a2724508d7d None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:30.656476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4e90d-707c-469a-a10c-9a2724508d7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:30.656779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4e90d-707c-469a-a10c-9a2724508d7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:30.658149 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4e90d-707c-469a-a10c-9a2724508d7d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:30.669182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 422/856] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:30.670409 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43964e-b4ba-4b45-bd79-e93be99d1516 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:30.670736 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43964e-b4ba-4b45-bd79-e93be99d1516 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:30.671015 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43964e-b4ba-4b45-bd79-e93be99d1516 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:30.672211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 423/857] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:30.672890 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43964e-b4ba-4b45-bd79-e93be99d1516 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:30.692444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8792a6b-578e-4793-8c4e-017ea6d67d2d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:30.727126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86a5b9-3368-49e5-acb5-16e2add63633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:30.727793 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86a5b9-3368-49e5-acb5-16e2add63633 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:30.728074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86a5b9-3368-49e5-acb5-16e2add63633 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:30.728352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86a5b9-3368-49e5-acb5-16e2add63633 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:30.729044 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86a5b9-3368-49e5-acb5-16e2add63633 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:30.729348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86a5b9-3368-49e5-acb5-16e2add63633 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:30.730726 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86a5b9-3368-49e5-acb5-16e2add63633 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:30.735093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 435/858] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:46:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:30.735854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86a5b9-3368-49e5-acb5-16e2add63633 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:30.782091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43964e-b4ba-4b45-bd79-e93be99d1516 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:30.819447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:30.824103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:30.831233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8e4a4-1dcd-4b93-a9c6-1b57ad58d57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:30.832044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8e4a4-1dcd-4b93-a9c6-1b57ad58d57d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:30.832331 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8e4a4-1dcd-4b93-a9c6-1b57ad58d57d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:30.832608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8e4a4-1dcd-4b93-a9c6-1b57ad58d57d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:30.845579 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8e4a4-1dcd-4b93-a9c6-1b57ad58d57d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:30.847165 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8e4a4-1dcd-4b93-a9c6-1b57ad58d57d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:30.863413 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86a5b9-3368-49e5-acb5-16e2add63633 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:30.867257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-31c8e4a4-1dcd-4b93-a9c6-1b57ad58d57d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:30.867257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:30.867257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:30.867257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:30.867257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:30.867257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:30.867257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:30.867257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:30.867257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:30.867257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:30.867257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:30.867257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:30.867257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:30.867257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:30.867257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:30.868208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:30.868208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:30.868208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:30.868208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:30.868208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:30.868208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:30.868208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:30.868208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:30.868208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:30.868208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:30.868208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:30.868208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:30.868208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:30.868208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:30.868208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:30.868208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:30.868208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:30.869126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:30.869126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:30.869126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:30.869126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:30.869126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 424/859] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:30.869769 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc04287-bc16-4197-a97f-8ac1ce109fd5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:30.870060 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc04287-bc16-4197-a97f-8ac1ce109fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:30.870340 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc04287-bc16-4197-a97f-8ac1ce109fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:30.892049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2bc04287-bc16-4197-a97f-8ac1ce109fd5 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:30.893879 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2bc04287-bc16-4197-a97f-8ac1ce109fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:30.916464 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383663b-a611-434f-b910-1b37eea3c4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:30.917783 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383663b-a611-434f-b910-1b37eea3c4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:30.917783 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383663b-a611-434f-b910-1b37eea3c4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:30.917974 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383663b-a611-434f-b910-1b37eea3c4a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:30.925491 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383663b-a611-434f-b910-1b37eea3c4a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:30.926971 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383663b-a611-434f-b910-1b37eea3c4a4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:30.941863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-6383663b-a611-434f-b910-1b37eea3c4a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:30.941863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:30.941863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:30.941863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:30.941863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:30.941863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:30.941863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:30.941863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:30.941863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:30.941863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:30.941863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:30.941863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:30.941863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:30.941863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:30.941863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:30.942888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:30.942888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:30.942888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:30.942888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:30.942888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:30.942888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:30.942888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:30.942888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:30.942888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:30.942888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:30.942888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:30.942888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:30.942888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:30.942888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:30.942888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:30.942888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:30.942888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:30.945120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:30.945120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:30.945120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:30.945120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:30.945120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:30.945120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:30.945120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:30.945120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:30.948675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 436/860] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:30.949411 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383663b-a611-434f-b910-1b37eea3c4a4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:30.975873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 425/861] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:30.976863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc04287-bc16-4197-a97f-8ac1ce109fd5 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:31.073012 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383663b-a611-434f-b910-1b37eea3c4a4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:31.087283 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc04287-bc16-4197-a97f-8ac1ce109fd5 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:31.097300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:31.103728 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:31.111800 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f430b-5c9e-434e-af6d-b2236433fdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:31.112521 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f430b-5c9e-434e-af6d-b2236433fdc1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:31.112740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f430b-5c9e-434e-af6d-b2236433fdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:31.113011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f430b-5c9e-434e-af6d-b2236433fdc1 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:31.113726 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f430b-5c9e-434e-af6d-b2236433fdc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:31.114029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f430b-5c9e-434e-af6d-b2236433fdc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:31.115495 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f430b-5c9e-434e-af6d-b2236433fdc1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:31.125870 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af370dd2-df2d-4b65-9619-f6c37b615659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:31.126525 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af370dd2-df2d-4b65-9619-f6c37b615659 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:31.126835 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af370dd2-df2d-4b65-9619-f6c37b615659 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:31.127110 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af370dd2-df2d-4b65-9619-f6c37b615659 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:31.127889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af370dd2-df2d-4b65-9619-f6c37b615659 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:31.133074 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af370dd2-df2d-4b65-9619-f6c37b615659 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:31.133220 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 437/862] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:31.133220 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f430b-5c9e-434e-af6d-b2236433fdc1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:31.133388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af370dd2-df2d-4b65-9619-f6c37b615659 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:31.142932 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 426/863] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:31.144435 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af370dd2-df2d-4b65-9619-f6c37b615659 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:31.254461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f430b-5c9e-434e-af6d-b2236433fdc1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:31.264899 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af370dd2-df2d-4b65-9619-f6c37b615659 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:31.293075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1338c5d-5b62-4049-9ac0-cdcedf55b939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:31.293889 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1338c5d-5b62-4049-9ac0-cdcedf55b939 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:31.294173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1338c5d-5b62-4049-9ac0-cdcedf55b939 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:31.294444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1338c5d-5b62-4049-9ac0-cdcedf55b939 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:31.295133 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1338c5d-5b62-4049-9ac0-cdcedf55b939 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:31.296497 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1338c5d-5b62-4049-9ac0-cdcedf55b939 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:31.309289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-b1338c5d-5b62-4049-9ac0-cdcedf55b939 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:46:31.309289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:31.309289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:31.309289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:31.309289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:31.309289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:31.309289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:31.309289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:31.309289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:46:31.309289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:31.309289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:31.309289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:31.309289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:31.309289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:31.309289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:31.309289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:31.310339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:31.310339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:31.310339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:31.310339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:31.310339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:31.310339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:31.310339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:31.310339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:31.310339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:31.310339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:31.310339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:31.310339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:31.310339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:31.310339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:31.310339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:31.310339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:31.311285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:31.311285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:31.311285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:31.311285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:31.311285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:31.311285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:31.311285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:31.311285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:31.311285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:31.311285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:31.311285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:31.311285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:31.311285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:31.311285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:46:31.311285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:31.311285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 438/864] 10.222.0.70 () {56 vars in 1132 bytes} [Thu Jun 16 18:46:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:31.312206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1338c5d-5b62-4049-9ac0-cdcedf55b939 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:31.316358 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c37b79-cc13-4406-b1fc-797784d02f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:31.317431 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c37b79-cc13-4406-b1fc-797784d02f6d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:31.318663 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c37b79-cc13-4406-b1fc-797784d02f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:31.319153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c37b79-cc13-4406-b1fc-797784d02f6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:31.328573 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c37b79-cc13-4406-b1fc-797784d02f6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:31.330916 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c37b79-cc13-4406-b1fc-797784d02f6d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:31.351541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b8c37b79-cc13-4406-b1fc-797784d02f6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:31.351541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:31.351541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:31.351541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:31.351541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:31.351541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:31.351541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:31.351541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:31.351541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:31.351541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:31.351541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:31.351541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:31.351541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:31.351541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:31.351541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:31.352643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:31.352643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:31.352643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:31.352643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:31.352643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:31.352643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:31.352643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:31.352643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:31.352643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:31.352643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:31.352643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:31.352643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:31.352643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:31.352643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:31.352643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:31.352643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:31.352643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:31.353567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:31.353567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:31.353567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:31.353567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:31.353567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:31.353567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:31.353567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:31.353567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:31.355356 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 427/865] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:31.356275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c37b79-cc13-4406-b1fc-797784d02f6d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:31.377336 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:31.389907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:31.429581 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1338c5d-5b62-4049-9ac0-cdcedf55b939 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:31.465891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c37b79-cc13-4406-b1fc-797784d02f6d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:31.466730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244bb40-a349-4d1d-bdcf-45005daa0104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:31.467687 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244bb40-a349-4d1d-bdcf-45005daa0104 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:31.468233 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244bb40-a349-4d1d-bdcf-45005daa0104 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:31.468506 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244bb40-a349-4d1d-bdcf-45005daa0104 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:31.469333 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244bb40-a349-4d1d-bdcf-45005daa0104 None admin] RBAC: Adding query filter params (role.id=c123034c11cc4d9ea6cab00e05bdcd53, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=02c826fc0baa480393df4c884c9e35dc) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:31.469637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244bb40-a349-4d1d-bdcf-45005daa0104 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:31.471085 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244bb40-a349-4d1d-bdcf-45005daa0104 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:31.486672 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 439/866] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:46:31 2022] GET /identity/v3/role_assignments?user.id=02c826fc0baa480393df4c884c9e35dc&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=c123034c11cc4d9ea6cab00e05bdcd53 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:31.486672 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244bb40-a349-4d1d-bdcf-45005daa0104 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:31.525626 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f211cd5-b10e-4ef0-88cb-5704dca9b9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:31.526985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f211cd5-b10e-4ef0-88cb-5704dca9b9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:31.527403 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f211cd5-b10e-4ef0-88cb-5704dca9b9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:31.527681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f211cd5-b10e-4ef0-88cb-5704dca9b9a8 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:31.528440 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f211cd5-b10e-4ef0-88cb-5704dca9b9a8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:31.528774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f211cd5-b10e-4ef0-88cb-5704dca9b9a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:31.531405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f211cd5-b10e-4ef0-88cb-5704dca9b9a8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:31.538743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 428/867] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Jun 16 18:46:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:31.544498 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f211cd5-b10e-4ef0-88cb-5704dca9b9a8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:31.635662 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244bb40-a349-4d1d-bdcf-45005daa0104 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:31.654528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:31.662411 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:31.666546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f211cd5-b10e-4ef0-88cb-5704dca9b9a8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:31.684195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd89ce8-c358-4eca-be4d-12b7d4c95ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:31.684984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd89ce8-c358-4eca-be4d-12b7d4c95ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:31.685343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd89ce8-c358-4eca-be4d-12b7d4c95ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:31.685690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd89ce8-c358-4eca-be4d-12b7d4c95ea7 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:31.686562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd89ce8-c358-4eca-be4d-12b7d4c95ea7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:31.687218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd89ce8-c358-4eca-be4d-12b7d4c95ea7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:31.688456 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd89ce8-c358-4eca-be4d-12b7d4c95ea7 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:31.705634 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 440/868] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:31.709815 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942a2ff-32af-4270-ad59-0efbdddc3794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:31.710765 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942a2ff-32af-4270-ad59-0efbdddc3794 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:31.711331 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942a2ff-32af-4270-ad59-0efbdddc3794 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:31.711616 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942a2ff-32af-4270-ad59-0efbdddc3794 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:31.712014 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd89ce8-c358-4eca-be4d-12b7d4c95ea7 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:31.712329 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942a2ff-32af-4270-ad59-0efbdddc3794 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:31.712636 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942a2ff-32af-4270-ad59-0efbdddc3794 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:31.713988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942a2ff-32af-4270-ad59-0efbdddc3794 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:31.720573 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 429/869] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Jun 16 18:46:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:31.838409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd89ce8-c358-4eca-be4d-12b7d4c95ea7 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:31.880358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248a9c7-7434-412d-a776-42267822de9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:31.881612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248a9c7-7434-412d-a776-42267822de9d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:31.881856 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248a9c7-7434-412d-a776-42267822de9d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:31.882116 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248a9c7-7434-412d-a776-42267822de9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:31.882993 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248a9c7-7434-412d-a776-42267822de9d None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=7b22fe0ab0354385924923d0d443acae) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:31.883307 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248a9c7-7434-412d-a776-42267822de9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:31.884763 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248a9c7-7434-412d-a776-42267822de9d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:31.892004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 441/870] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:46:31 2022] GET /identity/v3/role_assignments?user.id=7b22fe0ab0354385924923d0d443acae&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:32.948293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7905f1f-b904-4d17-b97c-089430bd3a2a None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:32.948815 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7905f1f-b904-4d17-b97c-089430bd3a2a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:32.949128 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7905f1f-b904-4d17-b97c-089430bd3a2a None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:32.950896 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 430/871] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:32.957893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3df68-aaeb-4001-a52d-acd2046325c2 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:32.957893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3df68-aaeb-4001-a52d-acd2046325c2 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:32.957893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3df68-aaeb-4001-a52d-acd2046325c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:32.977205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b9f3df68-aaeb-4001-a52d-acd2046325c2 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:32.978796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b9f3df68-aaeb-4001-a52d-acd2046325c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:33.020964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:33.065733 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 442/872] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:46:33.089057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7905f1f-b904-4d17-b97c-089430bd3a2a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:33.197025 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7905f1f-b904-4d17-b97c-089430bd3a2a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:33.236803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cbf88d-2416-4e7e-a8d7-dff1f5f98d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:33.237433 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbf88d-2416-4e7e-a8d7-dff1f5f98d97 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:33.237736 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbf88d-2416-4e7e-a8d7-dff1f5f98d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:33.238021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbf88d-2416-4e7e-a8d7-dff1f5f98d97 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:33.238807 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cbf88d-2416-4e7e-a8d7-dff1f5f98d97 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:33.240242 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cbf88d-2416-4e7e-a8d7-dff1f5f98d97 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:33.256931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 431/873] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Jun 16 18:46:33 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:46:33.918850 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a7566-b7ae-46da-927a-c3c334eec594 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:33.919255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a7566-b7ae-46da-927a-c3c334eec594 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:33.919445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a7566-b7ae-46da-927a-c3c334eec594 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:33.921036 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 443/874] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:33.926194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416eca5-7f40-459e-b3a7-53cfd1b7264f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:33.926488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416eca5-7f40-459e-b3a7-53cfd1b7264f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:33.926792 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416eca5-7f40-459e-b3a7-53cfd1b7264f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:33.946119 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3416eca5-7f40-459e-b3a7-53cfd1b7264f None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:33.946119 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3416eca5-7f40-459e-b3a7-53cfd1b7264f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:33.955793 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c7f75-f270-4048-8ab2-e54961297505 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:33.956074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c7f75-f270-4048-8ab2-e54961297505 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:33.956350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c7f75-f270-4048-8ab2-e54961297505 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:33.960133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:33.964197 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 444/875] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:46:33.970180 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474b137-c9d9-4c00-be87-51f5417c5813 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:33.970465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474b137-c9d9-4c00-be87-51f5417c5813 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:33.970783 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474b137-c9d9-4c00-be87-51f5417c5813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:33.991245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-8474b137-c9d9-4c00-be87-51f5417c5813 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:33.992459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8474b137-c9d9-4c00-be87-51f5417c5813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:34.055760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 432/876] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:46:34.081970 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416eca5-7f40-459e-b3a7-53cfd1b7264f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:34.084697 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 445/877] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:46:34.108174 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474b137-c9d9-4c00-be87-51f5417c5813 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:34.204157 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416eca5-7f40-459e-b3a7-53cfd1b7264f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:34.231869 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474b137-c9d9-4c00-be87-51f5417c5813 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:34.233514 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:34.240659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b78ea7-97ab-4aa4-8246-6e832c42c5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:34.241367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b78ea7-97ab-4aa4-8246-6e832c42c5c9 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:34.241811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b78ea7-97ab-4aa4-8246-6e832c42c5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:34.242106 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b78ea7-97ab-4aa4-8246-6e832c42c5c9 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:34.242857 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b78ea7-97ab-4aa4-8246-6e832c42c5c9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:34.244397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b78ea7-97ab-4aa4-8246-6e832c42c5c9 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:34.260661 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 433/878] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Jun 16 18:46:34 2022] POST /identity/v3/services => generated 269 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:46:34.274644 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11364f98-aa71-4029-a0f8-af33d02d2e82 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:34.274644 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11364f98-aa71-4029-a0f8-af33d02d2e82 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:34.274644 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11364f98-aa71-4029-a0f8-af33d02d2e82 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:34.274644 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 434/879] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:34.278900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b3124-e277-4756-b769-8bcb2203a7e8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:34.279190 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b3124-e277-4756-b769-8bcb2203a7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:34.279490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b3124-e277-4756-b769-8bcb2203a7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:34.280873 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2b183-e1e2-4ca1-840c-d6c22d92d8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:34.281923 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2b183-e1e2-4ca1-840c-d6c22d92d8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:34.282367 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2b183-e1e2-4ca1-840c-d6c22d92d8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:34.282825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2b183-e1e2-4ca1-840c-d6c22d92d8e1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:34.284862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:34.296486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e2b183-e1e2-4ca1-840c-d6c22d92d8e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:34.302080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d54b3124-e277-4756-b769-8bcb2203a7e8 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:34.304983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d54b3124-e277-4756-b769-8bcb2203a7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:34.305339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-e7e2b183-e1e2-4ca1-840c-d6c22d92d8e1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:46:34.305339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:34.305339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:34.305339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:34.305339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:34.305339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:34.305339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:34.305339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:34.305339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:34.305339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:34.305339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:34.305339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:34.305339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:34.305339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:34.305339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:34.305339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:34.306339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:34.306339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:34.306339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:34.306339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:34.306339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:34.306339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:34.306339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:34.306339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:34.306339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:34.306339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:34.306339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:34.306339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:34.306339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:34.306339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:34.306339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:34.306339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:34.307278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:34.307278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:34.307278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:34.307278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:34.307278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:34.307278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:34.307278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:34.307278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:34.307278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:34.307278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:34.307278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:46:34.307278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:34.311666 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 446/880] 10.222.0.70 () {56 vars in 1120 bytes} [Thu Jun 16 18:46:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:34.312487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2b183-e1e2-4ca1-840c-d6c22d92d8e1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:34.406626 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 435/881] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:46:34.431773 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b3124-e277-4756-b769-8bcb2203a7e8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:34.461875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2b183-e1e2-4ca1-840c-d6c22d92d8e1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:34.504006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2539a-44cb-497d-84e1-9336b5c8a0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:34.506590 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2539a-44cb-497d-84e1-9336b5c8a0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:34.507083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2539a-44cb-497d-84e1-9336b5c8a0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:34.507545 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2539a-44cb-497d-84e1-9336b5c8a0a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:34.508491 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2539a-44cb-497d-84e1-9336b5c8a0a2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:34.508796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2539a-44cb-497d-84e1-9336b5c8a0a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:34.510164 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2539a-44cb-497d-84e1-9336b5c8a0a2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:34.519589 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 447/882] 10.222.0.70 () {56 vars in 1115 bytes} [Thu Jun 16 18:46:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:34.527296 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2539a-44cb-497d-84e1-9336b5c8a0a2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:34.554740 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b3124-e277-4756-b769-8bcb2203a7e8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:34.555570 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:34.587757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:34.593711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93ad5a-2d24-488b-ba1c-eced89f45d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:34.594845 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93ad5a-2d24-488b-ba1c-eced89f45d30 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:34.595552 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93ad5a-2d24-488b-ba1c-eced89f45d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:34.595835 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93ad5a-2d24-488b-ba1c-eced89f45d30 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:34.596770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93ad5a-2d24-488b-ba1c-eced89f45d30 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:34.598881 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93ad5a-2d24-488b-ba1c-eced89f45d30 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:34.615839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-df93ad5a-2d24-488b-ba1c-eced89f45d30 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:46:34.615839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:34.615839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:34.615839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:34.615839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:34.615839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:34.615839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:34.615839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:34.615839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:46:34.615839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:34.615839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:34.615839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:34.615839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:34.615839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:34.615839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:34.615839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:34.615839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:34.616915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:34.616915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:34.616915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:34.616915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:34.616915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:34.616915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:34.616915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:34.616915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:34.616915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:34.616915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:34.616915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:34.616915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:34.616915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:34.616915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:34.616915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:34.616915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:34.617768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:34.617768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:34.617768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:34.617768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:34.617768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:34.617768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:34.617768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:34.617768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:34.617768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:34.617768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:34.617768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:34.617768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:34.617768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:46:34.617768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:34.618515 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 436/883] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Jun 16 18:46:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:34.627671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93ad5a-2d24-488b-ba1c-eced89f45d30 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:34.650539 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2539a-44cb-497d-84e1-9336b5c8a0a2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:34.689816 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7265c-5225-4ea9-aec3-df25a4b5a2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:34.690975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7265c-5225-4ea9-aec3-df25a4b5a2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:34.691509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7265c-5225-4ea9-aec3-df25a4b5a2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:34.692097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7265c-5225-4ea9-aec3-df25a4b5a2b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:34.704159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7265c-5225-4ea9-aec3-df25a4b5a2b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:34.706099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7265c-5225-4ea9-aec3-df25a4b5a2b5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:34.726352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-0fc7265c-5225-4ea9-aec3-df25a4b5a2b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:34.726352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:34.726352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:34.726352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:34.726352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:34.726352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:34.726352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:34.726352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:34.726352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:34.726352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:34.726352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:34.726352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:34.726352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:34.726352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:34.726352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:34.727321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:34.727321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:34.727321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:34.727321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:34.727321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:34.727321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:34.727321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:34.727321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:34.727321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:34.727321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:34.727321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:34.727321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:34.727321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:34.727321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:34.727321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:34.727321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:34.727321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:34.728221 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:34.728221 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:34.728221 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:34.728221 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:34.734619 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93ad5a-2d24-488b-ba1c-eced89f45d30 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:34.734897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 448/884] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:34.736474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7265c-5225-4ea9-aec3-df25a4b5a2b5 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:34.783288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f165656-cf2c-49ff-8f4d-1d65c1870599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:34.783541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f165656-cf2c-49ff-8f4d-1d65c1870599 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:34.783809 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f165656-cf2c-49ff-8f4d-1d65c1870599 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:34.784070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f165656-cf2c-49ff-8f4d-1d65c1870599 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:34.785142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f165656-cf2c-49ff-8f4d-1d65c1870599 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:34.785700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f165656-cf2c-49ff-8f4d-1d65c1870599 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:34.787760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f165656-cf2c-49ff-8f4d-1d65c1870599 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:34.797681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 437/885] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Jun 16 18:46:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:34.798761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f165656-cf2c-49ff-8f4d-1d65c1870599 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:34.829340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:34.873109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7265c-5225-4ea9-aec3-df25a4b5a2b5 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:34.882842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:34.913470 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f750668-a38d-4267-9307-549f898fd8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:34.914522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f750668-a38d-4267-9307-549f898fd8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:34.915132 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f750668-a38d-4267-9307-549f898fd8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:34.915687 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f750668-a38d-4267-9307-549f898fd8e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:34.916715 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f750668-a38d-4267-9307-549f898fd8e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:34.917220 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f750668-a38d-4267-9307-549f898fd8e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:34.919000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f750668-a38d-4267-9307-549f898fd8e3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:34.924871 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f165656-cf2c-49ff-8f4d-1d65c1870599 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:34.938226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 449/886] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:34.939546 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f750668-a38d-4267-9307-549f898fd8e3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:34.970089 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a64b2-7fb7-4f50-82fe-622ede1a7c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:34.970799 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a64b2-7fb7-4f50-82fe-622ede1a7c29 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:34.971107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a64b2-7fb7-4f50-82fe-622ede1a7c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:34.971383 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a64b2-7fb7-4f50-82fe-622ede1a7c29 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:34.972091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a64b2-7fb7-4f50-82fe-622ede1a7c29 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:34.972392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a64b2-7fb7-4f50-82fe-622ede1a7c29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:34.973793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a64b2-7fb7-4f50-82fe-622ede1a7c29 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:34.986696 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 438/887] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Jun 16 18:46:34 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:35.064609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f750668-a38d-4267-9307-549f898fd8e3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:35.097943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:35.105020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771f3cd-d765-4412-a813-5b23435de485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:35.105854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771f3cd-d765-4412-a813-5b23435de485 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:35.106133 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771f3cd-d765-4412-a813-5b23435de485 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:35.106407 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771f3cd-d765-4412-a813-5b23435de485 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:35.114827 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771f3cd-d765-4412-a813-5b23435de485 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:35.116454 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771f3cd-d765-4412-a813-5b23435de485 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:35.134922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-d771f3cd-d765-4412-a813-5b23435de485 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:46:35.134922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:35.134922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:35.134922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:35.134922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:35.134922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:35.134922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:35.134922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:35.134922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:35.134922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:35.134922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:35.134922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:35.134922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:35.134922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:35.134922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:35.134922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:35.135975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:35.136938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:35.136938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:35.136938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:35.136938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:35.136938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:35.136938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:35.136938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:35.136938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:35.136938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:46:35.136938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:35.136938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 450/888] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Jun 16 18:46:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:35.147211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a64b2-7fb7-4f50-82fe-622ede1a7c29 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:35.265988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a64b2-7fb7-4f50-82fe-622ede1a7c29 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:35.291391 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:35.301771 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee788a7-6339-4f1c-9eb3-d3225b5cc97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:35.303024 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee788a7-6339-4f1c-9eb3-d3225b5cc97b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:35.303307 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee788a7-6339-4f1c-9eb3-d3225b5cc97b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:35.303579 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee788a7-6339-4f1c-9eb3-d3225b5cc97b None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:35.304465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee788a7-6339-4f1c-9eb3-d3225b5cc97b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:35.304771 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee788a7-6339-4f1c-9eb3-d3225b5cc97b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:35.306182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee788a7-6339-4f1c-9eb3-d3225b5cc97b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:35.326725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 439/889] 10.222.0.70 () {56 vars in 1149 bytes} [Thu Jun 16 18:46:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:35.330042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771f3cd-d765-4412-a813-5b23435de485 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:35.446281 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771f3cd-d765-4412-a813-5b23435de485 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:35.484550 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568ea78-2e8d-41fe-93b7-047d396d7c50 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:35.484875 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568ea78-2e8d-41fe-93b7-047d396d7c50 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:35.485122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568ea78-2e8d-41fe-93b7-047d396d7c50 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:35.485898 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d384a-5443-4376-bd32-eeb97ac57443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:35.486905 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 440/890] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:35.487151 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d384a-5443-4376-bd32-eeb97ac57443 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:35.487636 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d384a-5443-4376-bd32-eeb97ac57443 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:35.488119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d384a-5443-4376-bd32-eeb97ac57443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:35.493694 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09491e-0732-4cfa-a4e7-24c1251e0276 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:35.493694 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09491e-0732-4cfa-a4e7-24c1251e0276 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:35.493956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09491e-0732-4cfa-a4e7-24c1251e0276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:35.510914 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9f09491e-0732-4cfa-a4e7-24c1251e0276 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:35.512101 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9f09491e-0732-4cfa-a4e7-24c1251e0276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:35.564499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:35.577360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d384a-5443-4376-bd32-eeb97ac57443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:35.579380 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d384a-5443-4376-bd32-eeb97ac57443 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:35.606450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 441/891] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:46:35.633381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09491e-0732-4cfa-a4e7-24c1251e0276 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:35.691402 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 451/892] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:46:35 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:46:35.697183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d384a-5443-4376-bd32-eeb97ac57443 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:35.758157 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09491e-0732-4cfa-a4e7-24c1251e0276 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:35.774090 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:35.799198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4dff18-a126-43c0-80f9-cf82f5b75960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:35.800006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dff18-a126-43c0-80f9-cf82f5b75960 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:35.800394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dff18-a126-43c0-80f9-cf82f5b75960 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:35.800675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dff18-a126-43c0-80f9-cf82f5b75960 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:35.801326 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4dff18-a126-43c0-80f9-cf82f5b75960 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:35.802823 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4dff18-a126-43c0-80f9-cf82f5b75960 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:35.803881 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:35.813784 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d384a-5443-4376-bd32-eeb97ac57443 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:35.817586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-1f4dff18-a126-43c0-80f9-cf82f5b75960 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:46:35.817586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:35.817586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:35.817586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:35.817586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:35.817586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:35.817586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:35.817586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:35.817586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:46:35.817586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:35.817586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:35.817586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:35.817586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:35.817586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:35.817586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:35.817586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:35.817586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:35.819822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:35.819822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:35.819822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:35.819822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:35.819822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:35.819822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:35.819822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:35.819822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:35.819822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:35.819822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:35.819822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:35.819822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:35.819822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:35.819822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:35.819822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:35.819822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:35.820667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:35.820667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:35.820667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:35.820667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:35.820667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:35.820667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:35.820667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:35.820667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:35.820667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:35.820667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:35.820667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:35.820667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:35.820667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:46:35.820667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:35.820667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 442/893] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:46:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:35.827357 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4dff18-a126-43c0-80f9-cf82f5b75960 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:35.853743 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d838a3-7310-41d7-891a-4f0f608f18af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:35.854540 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d838a3-7310-41d7-891a-4f0f608f18af None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:35.854889 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d838a3-7310-41d7-891a-4f0f608f18af None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:35.855395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d838a3-7310-41d7-891a-4f0f608f18af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:35.870443 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d838a3-7310-41d7-891a-4f0f608f18af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:35.872093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d838a3-7310-41d7-891a-4f0f608f18af None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:35.891950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-04d838a3-7310-41d7-891a-4f0f608f18af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:35.891950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:35.891950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:35.891950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:35.891950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:35.891950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:35.891950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:35.891950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:35.891950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:35.891950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:35.891950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:35.891950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:35.891950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:35.891950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:35.891950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:35.893166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:35.893166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:35.893166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:35.893166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:35.893166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:35.893166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:35.893166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:35.893166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:35.893166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:35.893166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:35.893166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:35.893166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:35.893166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:35.893166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:35.893166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:35.893166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:35.893166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:35.894061 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:35.894061 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:35.894061 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:35.894061 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:35.898124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 452/894] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:35.899417 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d838a3-7310-41d7-891a-4f0f608f18af None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:35.981023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4dff18-a126-43c0-80f9-cf82f5b75960 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:36.024033 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea78e6-273c-4d43-9bee-ca3778550ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:36.025227 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea78e6-273c-4d43-9bee-ca3778550ceb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:36.025975 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea78e6-273c-4d43-9bee-ca3778550ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:36.026724 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea78e6-273c-4d43-9bee-ca3778550ceb None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:36.027606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea78e6-273c-4d43-9bee-ca3778550ceb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:36.028083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea78e6-273c-4d43-9bee-ca3778550ceb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:36.028688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d838a3-7310-41d7-891a-4f0f608f18af None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:36.029820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea78e6-273c-4d43-9bee-ca3778550ceb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:36.036288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 443/895] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:46:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:36.042968 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea78e6-273c-4d43-9bee-ca3778550ceb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:36.076601 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e03d4-c876-463f-973a-3c396e811206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:36.077203 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e03d4-c876-463f-973a-3c396e811206 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:36.077850 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e03d4-c876-463f-973a-3c396e811206 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:36.078212 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e03d4-c876-463f-973a-3c396e811206 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:36.079443 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e03d4-c876-463f-973a-3c396e811206 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:36.079855 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e03d4-c876-463f-973a-3c396e811206 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:36.081853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e03d4-c876-463f-973a-3c396e811206 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:36.099984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:36.100321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 453/896] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:36.102644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e03d4-c876-463f-973a-3c396e811206 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:36.107264 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:36.168497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea78e6-273c-4d43-9bee-ca3778550ceb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:36.210302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080a4e5-ac68-4079-b46f-f0dc575049cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:36.211100 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080a4e5-ac68-4079-b46f-f0dc575049cc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:36.211695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080a4e5-ac68-4079-b46f-f0dc575049cc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:36.211804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080a4e5-ac68-4079-b46f-f0dc575049cc None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:36.212590 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080a4e5-ac68-4079-b46f-f0dc575049cc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:36.212955 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080a4e5-ac68-4079-b46f-f0dc575049cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:36.214408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080a4e5-ac68-4079-b46f-f0dc575049cc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:36.215190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e03d4-c876-463f-973a-3c396e811206 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:36.225443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 444/897] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:46:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:36.227554 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080a4e5-ac68-4079-b46f-f0dc575049cc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:36.258110 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469d373-18bc-46af-9786-bb19612cc6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:36.259383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469d373-18bc-46af-9786-bb19612cc6de None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:36.259737 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469d373-18bc-46af-9786-bb19612cc6de None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:36.260080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469d373-18bc-46af-9786-bb19612cc6de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:36.269472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469d373-18bc-46af-9786-bb19612cc6de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:36.271345 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469d373-18bc-46af-9786-bb19612cc6de None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:36.287903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-9469d373-18bc-46af-9786-bb19612cc6de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:36.287903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:36.287903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:36.287903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:36.287903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:36.287903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:36.287903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:36.287903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:36.287903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:36.287903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:36.287903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:36.287903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:36.287903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:36.287903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:36.287903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:36.288916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:36.288916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:36.288916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:36.288916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:36.288916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:36.288916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:36.288916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:36.288916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:36.288916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:36.288916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:36.288916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:36.288916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:36.288916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:36.288916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:36.288916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:36.288916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:36.288916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:36.289815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:36.289815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:36.289815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:36.289815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:36.289815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:36.289815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:36.289815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:36.289815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:36.296838 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 454/898] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:36.298228 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469d373-18bc-46af-9786-bb19612cc6de None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:36.358978 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080a4e5-ac68-4079-b46f-f0dc575049cc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:36.362405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:36.373785 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:36.399364 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a233255-9bde-451a-a011-d54ecb3ff141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:36.400424 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a233255-9bde-451a-a011-d54ecb3ff141 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:36.400971 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a233255-9bde-451a-a011-d54ecb3ff141 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:36.401456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a233255-9bde-451a-a011-d54ecb3ff141 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:36.402667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a233255-9bde-451a-a011-d54ecb3ff141 None admin] RBAC: Adding query filter params (interface=public, service_id=9148691bac4640998721f5c08af51966, region_id=RegionOne) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:36.403192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a233255-9bde-451a-a011-d54ecb3ff141 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:36.404741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a233255-9bde-451a-a011-d54ecb3ff141 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:36.411052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 445/899] 10.222.0.70 () {56 vars in 1251 bytes} [Thu Jun 16 18:46:36 2022] GET /identity/v3/endpoints?service_id=9148691bac4640998721f5c08af51966&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:36.417466 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a233255-9bde-451a-a011-d54ecb3ff141 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:36.428016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469d373-18bc-46af-9786-bb19612cc6de None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:36.470873 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023edbd7-7fd6-4db2-8ad4-d66b7fadaab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:36.471530 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023edbd7-7fd6-4db2-8ad4-d66b7fadaab8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:36.471807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023edbd7-7fd6-4db2-8ad4-d66b7fadaab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:36.472081 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023edbd7-7fd6-4db2-8ad4-d66b7fadaab8 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:36.473262 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023edbd7-7fd6-4db2-8ad4-d66b7fadaab8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:36.473570 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023edbd7-7fd6-4db2-8ad4-d66b7fadaab8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:36.475560 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023edbd7-7fd6-4db2-8ad4-d66b7fadaab8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:36.490985 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 455/900] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:36.501406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023edbd7-7fd6-4db2-8ad4-d66b7fadaab8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:36.564589 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a233255-9bde-451a-a011-d54ecb3ff141 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:36.607044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ddea3d-a149-4efd-8373-ba7881f88d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:36.607783 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddea3d-a149-4efd-8373-ba7881f88d91 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:36.608063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddea3d-a149-4efd-8373-ba7881f88d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:36.608366 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddea3d-a149-4efd-8373-ba7881f88d91 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:36.609099 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ddea3d-a149-4efd-8373-ba7881f88d91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:36.610409 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ddea3d-a149-4efd-8373-ba7881f88d91 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:36.622596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 446/901] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Jun 16 18:46:36 2022] GET /identity/v3/services => generated 1268 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:46:36.622596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b2f7f-5638-4ab2-a280-c332f9f8ddc8 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:36.622596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b2f7f-5638-4ab2-a280-c332f9f8ddc8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:36.622596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b2f7f-5638-4ab2-a280-c332f9f8ddc8 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:36.622596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 447/902] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:36.625210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdf4bd-5c02-4c9b-a5eb-88061e014c8d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:36.625502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdf4bd-5c02-4c9b-a5eb-88061e014c8d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:36.625783 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdf4bd-5c02-4c9b-a5eb-88061e014c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:36.644903 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-febdf4bd-5c02-4c9b-a5eb-88061e014c8d None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:36.647471 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-febdf4bd-5c02-4c9b-a5eb-88061e014c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:36.651714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023edbd7-7fd6-4db2-8ad4-d66b7fadaab8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:36.681444 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:36.685180 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:36.696997 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb5e75-c7da-404e-888e-6db5656f9535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:36.697822 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb5e75-c7da-404e-888e-6db5656f9535 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:36.698198 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb5e75-c7da-404e-888e-6db5656f9535 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:36.698544 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb5e75-c7da-404e-888e-6db5656f9535 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:36.699487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb5e75-c7da-404e-888e-6db5656f9535 None admin] RBAC: Adding query filter params (role.id=ed8734ab73344d1983b1cdaa7fa3238a, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=02c826fc0baa480393df4c884c9e35dc) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:36.699875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb5e75-c7da-404e-888e-6db5656f9535 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:36.701449 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb5e75-c7da-404e-888e-6db5656f9535 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:36.708050 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 456/903] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:46:36 2022] GET /identity/v3/role_assignments?user.id=02c826fc0baa480393df4c884c9e35dc&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=ed8734ab73344d1983b1cdaa7fa3238a => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:36.744381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 448/904] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:46:36.763440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb5e75-c7da-404e-888e-6db5656f9535 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:36.897691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb5e75-c7da-404e-888e-6db5656f9535 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:36.933788 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182601ce-5cf8-4a3c-9905-3b61ac646040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:36.934612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182601ce-5cf8-4a3c-9905-3b61ac646040 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:36.934896 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182601ce-5cf8-4a3c-9905-3b61ac646040 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:36.935168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182601ce-5cf8-4a3c-9905-3b61ac646040 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:36.935817 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182601ce-5cf8-4a3c-9905-3b61ac646040 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:36.937166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182601ce-5cf8-4a3c-9905-3b61ac646040 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:36.954615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-182601ce-5cf8-4a3c-9905-3b61ac646040 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:46:36.954615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:36.954615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:36.954615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:36.954615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:36.954615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:36.954615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:36.954615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:36.954615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:46:36.954615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:36.954615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:36.954615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:36.954615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:36.954615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:36.954615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:36.954615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:36.955675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:36.955675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:36.955675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:36.955675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:36.955675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:36.955675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:36.955675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:36.955675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:36.955675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:36.955675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:36.955675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:36.955675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:36.955675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:36.955675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:36.955675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:36.955675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:36.956512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:36.956512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:36.956512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:36.956512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:36.956512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:36.956512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:36.956512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:36.956512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:36.956512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:36.956512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:36.956512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:36.956512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:36.956512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:36.956512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:46:36.956512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:36.962143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 457/905] 10.222.0.70 () {56 vars in 1132 bytes} [Thu Jun 16 18:46:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:36.962216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdf4bd-5c02-4c9b-a5eb-88061e014c8d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:37.069591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdf4bd-5c02-4c9b-a5eb-88061e014c8d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:37.106162 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744e160-9bd2-4716-b530-2e9dd253f816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:37.106860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e160-9bd2-4716-b530-2e9dd253f816 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:37.107141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e160-9bd2-4716-b530-2e9dd253f816 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:37.107413 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e160-9bd2-4716-b530-2e9dd253f816 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:37.108116 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744e160-9bd2-4716-b530-2e9dd253f816 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:37.108417 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744e160-9bd2-4716-b530-2e9dd253f816 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:37.109775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744e160-9bd2-4716-b530-2e9dd253f816 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:37.118680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 449/906] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Jun 16 18:46:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:37.120735 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182601ce-5cf8-4a3c-9905-3b61ac646040 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:37.180074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:37.235341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182601ce-5cf8-4a3c-9905-3b61ac646040 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:37.272416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87a5ef-33b9-41c0-9442-571188dd9218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:37.273080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87a5ef-33b9-41c0-9442-571188dd9218 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:37.273361 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87a5ef-33b9-41c0-9442-571188dd9218 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:37.273640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87a5ef-33b9-41c0-9442-571188dd9218 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:37.274337 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87a5ef-33b9-41c0-9442-571188dd9218 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:37.274678 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87a5ef-33b9-41c0-9442-571188dd9218 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:37.276040 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87a5ef-33b9-41c0-9442-571188dd9218 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:37.284976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 458/907] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Jun 16 18:46:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:37.287374 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d682ce-0b7e-4be4-a7f1-9b90fd193c78 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:37.288195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d682ce-0b7e-4be4-a7f1-9b90fd193c78 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:37.288195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d682ce-0b7e-4be4-a7f1-9b90fd193c78 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:37.289893 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 450/908] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:37.290812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87a5ef-33b9-41c0-9442-571188dd9218 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:37.296948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4deff-a2cc-4f57-8a76-e71a0e40b444 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:37.297684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4deff-a2cc-4f57-8a76-e71a0e40b444 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:37.298374 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4deff-a2cc-4f57-8a76-e71a0e40b444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:37.325724 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-67f4deff-a2cc-4f57-8a76-e71a0e40b444 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:37.328194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-67f4deff-a2cc-4f57-8a76-e71a0e40b444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:37.364454 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:37.404472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87a5ef-33b9-41c0-9442-571188dd9218 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:37.435079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:37.444669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288b5e0-a206-4675-ba7b-c460f21273de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:37.445692 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288b5e0-a206-4675-ba7b-c460f21273de None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:37.446132 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288b5e0-a206-4675-ba7b-c460f21273de None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:37.446565 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288b5e0-a206-4675-ba7b-c460f21273de None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:37.448370 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 451/909] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:46:37.448470 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288b5e0-a206-4675-ba7b-c460f21273de None admin] RBAC: Adding query filter params (interface=public, service_id=8b83c8369c1641ad9c3d974a4e676fa4, region_id=RegionOne) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:37.448470 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288b5e0-a206-4675-ba7b-c460f21273de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:37.449689 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288b5e0-a206-4675-ba7b-c460f21273de None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:37.455724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 459/910] 10.222.0.70 () {56 vars in 1251 bytes} [Thu Jun 16 18:46:37 2022] GET /identity/v3/endpoints?service_id=8b83c8369c1641ad9c3d974a4e676fa4&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:37.468589 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4deff-a2cc-4f57-8a76-e71a0e40b444 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:37.482219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288b5e0-a206-4675-ba7b-c460f21273de None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:37.575991 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4deff-a2cc-4f57-8a76-e71a0e40b444 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:37.612735 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ce634-a1d4-4a59-aa96-8c5895212112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:37.613527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ce634-a1d4-4a59-aa96-8c5895212112 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:37.613880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ce634-a1d4-4a59-aa96-8c5895212112 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:37.614238 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ce634-a1d4-4a59-aa96-8c5895212112 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:37.615731 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ce634-a1d4-4a59-aa96-8c5895212112 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:37.617497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ce634-a1d4-4a59-aa96-8c5895212112 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:37.617898 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288b5e0-a206-4675-ba7b-c460f21273de None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:37.631406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 452/911] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Jun 16 18:46:37 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:46:37.659581 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29a8dc-a5f1-4d2e-92b4-7fedc255d589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:37.660585 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29a8dc-a5f1-4d2e-92b4-7fedc255d589 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:37.661047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29a8dc-a5f1-4d2e-92b4-7fedc255d589 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:37.661648 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29a8dc-a5f1-4d2e-92b4-7fedc255d589 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:37.662743 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b29a8dc-a5f1-4d2e-92b4-7fedc255d589 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:37.664362 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b29a8dc-a5f1-4d2e-92b4-7fedc255d589 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:37.682644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 460/912] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Jun 16 18:46:37 2022] POST /identity/v3/services => generated 241 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:46:38.940535 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc0b1a-28f5-45eb-a2ad-376aec295ed2 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:38.941108 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc0b1a-28f5-45eb-a2ad-376aec295ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:38.941108 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc0b1a-28f5-45eb-a2ad-376aec295ed2 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:38.943304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 453/913] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:38.951200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c226f6a-aefd-4ddc-8284-1e1a92186b6f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:38.952118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c226f6a-aefd-4ddc-8284-1e1a92186b6f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:38.952626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c226f6a-aefd-4ddc-8284-1e1a92186b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:38.971171 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-8c226f6a-aefd-4ddc-8284-1e1a92186b6f None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:38.972685 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8c226f6a-aefd-4ddc-8284-1e1a92186b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:39.004324 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bdf37-275c-432e-928f-da26befb2c03 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:39.005310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bdf37-275c-432e-928f-da26befb2c03 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:39.006084 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bdf37-275c-432e-928f-da26befb2c03 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:39.008372 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 454/914] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:39.010792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:39.014936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484554b6-3213-4078-a557-0bb5f4496795 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:39.015687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484554b6-3213-4078-a557-0bb5f4496795 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:39.016175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484554b6-3213-4078-a557-0bb5f4496795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:39.037247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-484554b6-3213-4078-a557-0bb5f4496795 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:39.039551 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-484554b6-3213-4078-a557-0bb5f4496795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:39.058221 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 461/915] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:39.060098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:39.084740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c226f6a-aefd-4ddc-8284-1e1a92186b6f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:39.122783 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 455/916] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:39.145593 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484554b6-3213-4078-a557-0bb5f4496795 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:39.191439 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c226f6a-aefd-4ddc-8284-1e1a92186b6f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:39.228030 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a64d43-424d-4312-b6f4-9127355b8321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:39.228831 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a64d43-424d-4312-b6f4-9127355b8321 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:39.229114 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a64d43-424d-4312-b6f4-9127355b8321 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:39.229390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a64d43-424d-4312-b6f4-9127355b8321 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:39.230029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a64d43-424d-4312-b6f4-9127355b8321 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:39.231591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a64d43-424d-4312-b6f4-9127355b8321 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:39.244478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-97a64d43-424d-4312-b6f4-9127355b8321 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:46:39.244478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:39.244478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:39.244478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:39.244478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:39.244478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:39.244478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:39.244478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:39.244478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:46:39.244478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:39.244478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:39.244478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:39.244478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:39.244478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:39.244478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:39.244478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:39.244478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:39.245505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:39.245505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:39.245505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:39.245505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:39.245505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:39.245505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:39.245505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:39.245505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:39.245505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:39.245505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:39.245505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:39.245505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:39.245505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:39.245505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:39.245505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:39.245505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:39.246408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:39.246408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:39.246408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:39.246408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:39.246408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:39.246408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:39.246408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:39.246408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:39.246408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:39.246408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:39.246408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:39.246408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:39.246408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:46:39.246408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:39.248922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 462/917] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:46:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:39.254988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484554b6-3213-4078-a557-0bb5f4496795 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:39.255282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a64d43-424d-4312-b6f4-9127355b8321 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:39.289014 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb60fc-d74f-407a-9146-310e6642c4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:39.290044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb60fc-d74f-407a-9146-310e6642c4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:39.290425 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb60fc-d74f-407a-9146-310e6642c4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:39.290843 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb60fc-d74f-407a-9146-310e6642c4c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:39.301853 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb60fc-d74f-407a-9146-310e6642c4c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:39.302497 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:39.310363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-95bb60fc-d74f-407a-9146-310e6642c4c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:46:39.310363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:39.310363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:39.310363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:39.310363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:39.310363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:39.310363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:39.310363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:39.310363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:39.310363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:39.310363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:39.310363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:39.310363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:39.310363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:39.310363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:39.310363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:39.311528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:39.311528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:39.311528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:39.311528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:39.311528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:39.311528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:39.311528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:39.311528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:39.311528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:39.311528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:39.311528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:39.311528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:39.311528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:39.311528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:39.311528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:39.311528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:39.312418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:39.312418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:39.312418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:39.312418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:39.312418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:39.312418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:39.312418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:39.312418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:39.312418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:39.312418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:39.312418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:46:39.312418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:39.318463 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 456/918] 10.222.0.70 () {56 vars in 1120 bytes} [Thu Jun 16 18:46:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:39.319466 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb60fc-d74f-407a-9146-310e6642c4c8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:39.332361 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:39.379879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a64d43-424d-4312-b6f4-9127355b8321 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:39.417848 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414a1db-eaf7-4967-a879-dff71d1a7874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:39.418986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414a1db-eaf7-4967-a879-dff71d1a7874 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:39.419517 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414a1db-eaf7-4967-a879-dff71d1a7874 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:39.420083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414a1db-eaf7-4967-a879-dff71d1a7874 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:39.420965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414a1db-eaf7-4967-a879-dff71d1a7874 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:39.421459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414a1db-eaf7-4967-a879-dff71d1a7874 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:39.423256 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414a1db-eaf7-4967-a879-dff71d1a7874 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:39.426129 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb60fc-d74f-407a-9146-310e6642c4c8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:39.429226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 463/919] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:46:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:39.437549 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414a1db-eaf7-4967-a879-dff71d1a7874 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:39.463916 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c071f6-6fe2-4648-b00e-5f116e22d62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:39.464605 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c071f6-6fe2-4648-b00e-5f116e22d62d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:39.465007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c071f6-6fe2-4648-b00e-5f116e22d62d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:39.465352 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c071f6-6fe2-4648-b00e-5f116e22d62d None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:39.466217 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c071f6-6fe2-4648-b00e-5f116e22d62d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:39.466601 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c071f6-6fe2-4648-b00e-5f116e22d62d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:39.468109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c071f6-6fe2-4648-b00e-5f116e22d62d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:39.482221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 457/920] 10.222.0.70 () {56 vars in 1115 bytes} [Thu Jun 16 18:46:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:39.483229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c071f6-6fe2-4648-b00e-5f116e22d62d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:39.557013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:39.564073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414a1db-eaf7-4967-a879-dff71d1a7874 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:39.566107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:39.600501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe588e8a-9fd2-492d-a07f-c2437d79e9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:39.601188 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe588e8a-9fd2-492d-a07f-c2437d79e9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:39.601608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe588e8a-9fd2-492d-a07f-c2437d79e9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:39.601729 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe588e8a-9fd2-492d-a07f-c2437d79e9c7 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:39.602482 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe588e8a-9fd2-492d-a07f-c2437d79e9c7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:39.602843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe588e8a-9fd2-492d-a07f-c2437d79e9c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:39.604210 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe588e8a-9fd2-492d-a07f-c2437d79e9c7 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:39.604996 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c071f6-6fe2-4648-b00e-5f116e22d62d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:39.614009 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 464/921] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Jun 16 18:46:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:39.615243 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe588e8a-9fd2-492d-a07f-c2437d79e9c7 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:39.646026 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba812c2d-00c3-47fb-93ba-3917b0f36184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:39.647401 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba812c2d-00c3-47fb-93ba-3917b0f36184 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:39.647869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba812c2d-00c3-47fb-93ba-3917b0f36184 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:39.648402 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba812c2d-00c3-47fb-93ba-3917b0f36184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:39.660591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba812c2d-00c3-47fb-93ba-3917b0f36184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:39.663342 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba812c2d-00c3-47fb-93ba-3917b0f36184 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:39.685161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ba812c2d-00c3-47fb-93ba-3917b0f36184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:39.685161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:39.685161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:39.685161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:39.685161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:39.685161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:39.685161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:39.685161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:39.685161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:39.685161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:39.685161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:39.685161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:39.685161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:39.685161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:39.685161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:39.686398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:39.686398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:39.686398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:39.686398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:39.686398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:39.686398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:39.686398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:39.686398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:39.686398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:39.686398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:39.686398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:39.686398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:39.686398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:39.686398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:39.686398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:39.686398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:39.686398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:39.687537 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:39.687537 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:39.687537 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:39.687537 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:39.694789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 458/922] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:39.695914 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba812c2d-00c3-47fb-93ba-3917b0f36184 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:39.726911 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe588e8a-9fd2-492d-a07f-c2437d79e9c7 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:39.762363 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7db50-73d5-40c5-bab5-7bb236cc0485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:39.763152 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7db50-73d5-40c5-bab5-7bb236cc0485 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:39.763501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7db50-73d5-40c5-bab5-7bb236cc0485 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:39.763850 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7db50-73d5-40c5-bab5-7bb236cc0485 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:39.764650 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7db50-73d5-40c5-bab5-7bb236cc0485 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:39.766066 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7db50-73d5-40c5-bab5-7bb236cc0485 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:39.799207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 465/923] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Jun 16 18:46:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:46:39.861014 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba812c2d-00c3-47fb-93ba-3917b0f36184 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:39.876543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3091c0-215a-40d9-b570-7ee7d9d36dee None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:39.876866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3091c0-215a-40d9-b570-7ee7d9d36dee None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:39.877104 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3091c0-215a-40d9-b570-7ee7d9d36dee None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:39.882796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 466/924] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:46:39.884609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c07af2-8127-43b8-a76a-ae871cbc060e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:39.884980 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c07af2-8127-43b8-a76a-ae871cbc060e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:39.885324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c07af2-8127-43b8-a76a-ae871cbc060e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:39.892940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:39.904071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-16c07af2-8127-43b8-a76a-ae871cbc060e None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:39.905330 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-16c07af2-8127-43b8-a76a-ae871cbc060e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:39.916237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:39.917233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d82a35-f375-4704-8c6c-016bd1255c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:39.917760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d82a35-f375-4704-8c6c-016bd1255c9a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:39.918400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d82a35-f375-4704-8c6c-016bd1255c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:39.918801 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d82a35-f375-4704-8c6c-016bd1255c9a None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:39.919930 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d82a35-f375-4704-8c6c-016bd1255c9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:39.920246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d82a35-f375-4704-8c6c-016bd1255c9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:39.922346 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d82a35-f375-4704-8c6c-016bd1255c9a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:39.937251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 459/925] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:39.938838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b9a3a-008f-4057-a65c-23dd4a3a0ca3 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:39.939127 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b9a3a-008f-4057-a65c-23dd4a3a0ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:39.939402 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b9a3a-008f-4057-a65c-23dd4a3a0ca3 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:39.940867 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 460/926] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:39.943225 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b9a3a-008f-4057-a65c-23dd4a3a0ca3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:39.995062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 467/927] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:39.996989 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea253c6-fa60-4526-aea9-7971a73c6000 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:39.997485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea253c6-fa60-4526-aea9-7971a73c6000 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:39.997757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea253c6-fa60-4526-aea9-7971a73c6000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:40.015987 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-7ea253c6-fa60-4526-aea9-7971a73c6000 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:40.017420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7ea253c6-fa60-4526-aea9-7971a73c6000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:40.073024 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b9a3a-008f-4057-a65c-23dd4a3a0ca3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:40.093482 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 468/928] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:40.094126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea253c6-fa60-4526-aea9-7971a73c6000 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:40.115893 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04879253-b895-4590-9f06-cd35cf782382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:40.117505 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04879253-b895-4590-9f06-cd35cf782382 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:40.118250 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04879253-b895-4590-9f06-cd35cf782382 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:40.118745 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04879253-b895-4590-9f06-cd35cf782382 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:40.130680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04879253-b895-4590-9f06-cd35cf782382 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:40.132836 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04879253-b895-4590-9f06-cd35cf782382 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:40.149604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-04879253-b895-4590-9f06-cd35cf782382 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:46:40.149604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:40.149604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:40.149604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:40.149604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:40.149604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:40.149604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:40.149604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:40.149604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:40.149604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:40.149604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:40.149604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:40.149604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:40.149604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:40.149604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:40.149604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:40.150606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:40.151559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:40.151559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:40.151559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:40.151559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:40.151559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:40.151559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:40.151559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:40.151559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:40.151559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:46:40.151559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:40.151559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 461/929] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Jun 16 18:46:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:40.151559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04879253-b895-4590-9f06-cd35cf782382 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:40.179112 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:40.198318 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:40.209515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea253c6-fa60-4526-aea9-7971a73c6000 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:40.247068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e4e4f-d6d4-4b34-b1db-861c548b02c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:40.247792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e4e4f-d6d4-4b34-b1db-861c548b02c3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:40.248074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e4e4f-d6d4-4b34-b1db-861c548b02c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:40.248398 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e4e4f-d6d4-4b34-b1db-861c548b02c3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:40.249048 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e4e4f-d6d4-4b34-b1db-861c548b02c3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:40.250403 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e4e4f-d6d4-4b34-b1db-861c548b02c3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:40.263354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04879253-b895-4590-9f06-cd35cf782382 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:40.264571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-488e4e4f-d6d4-4b34-b1db-861c548b02c3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:46:40.264571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:40.264571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:40.264571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:40.264571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:40.264571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:40.264571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:40.264571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:40.264571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:46:40.264571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:40.264571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:40.264571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:40.264571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:40.264571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:40.264571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:40.264571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:40.265493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:40.265493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:40.265493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:40.265493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:40.265493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:40.265493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:40.265493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:40.265493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:40.265493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:40.265493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:40.265493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:40.265493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:40.265493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:40.265493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:40.265493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:40.265493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:40.266359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:40.266359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:40.266359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:40.266359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:40.266359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:40.266359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:40.266359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:40.266359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:40.266359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:40.266359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:40.266359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:40.266359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:40.266359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:40.266359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:46:40.266359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:40.266359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 469/930] 10.222.0.70 () {56 vars in 1132 bytes} [Thu Jun 16 18:46:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:40.267540 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e4e4f-d6d4-4b34-b1db-861c548b02c3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:40.308570 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea868842-ea75-4233-a01c-77dcef257175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:40.309386 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea868842-ea75-4233-a01c-77dcef257175 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:40.309671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea868842-ea75-4233-a01c-77dcef257175 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:40.309951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea868842-ea75-4233-a01c-77dcef257175 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:40.310636 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea868842-ea75-4233-a01c-77dcef257175 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:40.312442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea868842-ea75-4233-a01c-77dcef257175 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:40.326860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ea868842-ea75-4233-a01c-77dcef257175 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:46:40.326860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:40.326860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:40.326860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:40.326860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:40.326860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:40.326860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:40.326860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:40.326860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:46:40.326860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:40.326860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:40.326860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:40.326860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:40.326860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:40.326860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:40.326860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:40.326860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:40.327902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:40.327902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:40.327902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:40.327902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:40.327902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:40.327902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:40.327902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:40.327902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:40.327902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:40.327902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:40.327902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:40.327902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:40.327902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:40.327902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:40.327902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:40.327902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:40.328776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:40.328776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:40.328776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:40.328776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:40.328776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:40.328776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:40.328776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:40.328776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:40.328776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:40.328776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:40.328776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:40.328776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:40.328776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:46:40.328776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:40.328776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 462/931] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Jun 16 18:46:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:40.329599 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea868842-ea75-4233-a01c-77dcef257175 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:40.388306 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e4e4f-d6d4-4b34-b1db-861c548b02c3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:40.426453 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30c814-c867-45d6-9f15-c18c0aca3b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:40.427235 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30c814-c867-45d6-9f15-c18c0aca3b12 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:40.427523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30c814-c867-45d6-9f15-c18c0aca3b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:40.427803 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30c814-c867-45d6-9f15-c18c0aca3b12 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:40.428634 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30c814-c867-45d6-9f15-c18c0aca3b12 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:40.429069 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30c814-c867-45d6-9f15-c18c0aca3b12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:40.430532 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30c814-c867-45d6-9f15-c18c0aca3b12 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:40.440088 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:40.440534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea868842-ea75-4233-a01c-77dcef257175 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:40.443153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:40.453060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 470/932] 10.222.0.70 () {56 vars in 1149 bytes} [Thu Jun 16 18:46:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:40.453894 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30c814-c867-45d6-9f15-c18c0aca3b12 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:40.480245 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b90269-6c87-4597-a30f-2181b93ba8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:40.480910 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b90269-6c87-4597-a30f-2181b93ba8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:40.481194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b90269-6c87-4597-a30f-2181b93ba8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:40.481471 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b90269-6c87-4597-a30f-2181b93ba8a5 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:40.482171 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b90269-6c87-4597-a30f-2181b93ba8a5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:40.482473 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b90269-6c87-4597-a30f-2181b93ba8a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:40.483870 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b90269-6c87-4597-a30f-2181b93ba8a5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:40.493444 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 463/933] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Jun 16 18:46:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:40.493444 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b90269-6c87-4597-a30f-2181b93ba8a5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:40.568273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30c814-c867-45d6-9f15-c18c0aca3b12 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:40.596790 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b90269-6c87-4597-a30f-2181b93ba8a5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:40.605973 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be72ba5-dc57-4d69-930c-8e0cd7776fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:40.606818 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be72ba5-dc57-4d69-930c-8e0cd7776fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:40.607107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be72ba5-dc57-4d69-930c-8e0cd7776fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:40.607386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be72ba5-dc57-4d69-930c-8e0cd7776fb8 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:40.608220 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be72ba5-dc57-4d69-930c-8e0cd7776fb8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:40.608535 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be72ba5-dc57-4d69-930c-8e0cd7776fb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:40.609902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be72ba5-dc57-4d69-930c-8e0cd7776fb8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:40.620201 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 471/934] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Jun 16 18:46:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:40.620201 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be72ba5-dc57-4d69-930c-8e0cd7776fb8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:40.637345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c750b73-2ffd-4c3c-94a7-9c34d36e244b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:40.638522 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c750b73-2ffd-4c3c-94a7-9c34d36e244b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:40.638979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c750b73-2ffd-4c3c-94a7-9c34d36e244b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:40.639266 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c750b73-2ffd-4c3c-94a7-9c34d36e244b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:40.650840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c750b73-2ffd-4c3c-94a7-9c34d36e244b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:40.652311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c750b73-2ffd-4c3c-94a7-9c34d36e244b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:40.669741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-7c750b73-2ffd-4c3c-94a7-9c34d36e244b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:40.669741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:40.669741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:40.669741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:40.669741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:40.669741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:40.669741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:40.669741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:40.669741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:40.669741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:40.669741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:40.669741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:40.669741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:40.669741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:40.669741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:40.670901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:40.670901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:40.670901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:40.670901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:40.670901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:40.670901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:40.670901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:40.670901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:40.670901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:40.670901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:40.670901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:40.670901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:40.670901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:40.670901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:40.670901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:40.670901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:40.670901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:40.671839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:40.671839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:40.671839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:40.671839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:40.671839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 464/935] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:40.672131 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c750b73-2ffd-4c3c-94a7-9c34d36e244b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:40.692309 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:40.697711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:40.737572 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be72ba5-dc57-4d69-930c-8e0cd7776fb8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:40.775043 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b636226-37e6-455d-b3c5-b8b9d6356515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:40.775710 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b636226-37e6-455d-b3c5-b8b9d6356515 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:40.775997 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b636226-37e6-455d-b3c5-b8b9d6356515 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:40.776258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b636226-37e6-455d-b3c5-b8b9d6356515 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:40.776946 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b636226-37e6-455d-b3c5-b8b9d6356515 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:40.777246 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b636226-37e6-455d-b3c5-b8b9d6356515 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:40.778610 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b636226-37e6-455d-b3c5-b8b9d6356515 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:40.788382 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 472/936] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Jun 16 18:46:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:40.788382 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b636226-37e6-455d-b3c5-b8b9d6356515 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:40.792060 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c750b73-2ffd-4c3c-94a7-9c34d36e244b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:40.827611 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea20f6e-965f-4851-8e96-0bdbd1e6e876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:40.828197 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea20f6e-965f-4851-8e96-0bdbd1e6e876 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:40.828477 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea20f6e-965f-4851-8e96-0bdbd1e6e876 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:40.828748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea20f6e-965f-4851-8e96-0bdbd1e6e876 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:40.829456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea20f6e-965f-4851-8e96-0bdbd1e6e876 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:40.829750 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea20f6e-965f-4851-8e96-0bdbd1e6e876 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:40.831135 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea20f6e-965f-4851-8e96-0bdbd1e6e876 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:40.836202 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 465/937] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Jun 16 18:46:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:40.836951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea20f6e-965f-4851-8e96-0bdbd1e6e876 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:40.899965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b636226-37e6-455d-b3c5-b8b9d6356515 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:40.930945 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:40.934192 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:40.948323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea20f6e-965f-4851-8e96-0bdbd1e6e876 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:40.952722 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893453b-b70b-4bed-ad5d-91de8d726edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:40.953386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893453b-b70b-4bed-ad5d-91de8d726edc None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:40.953662 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893453b-b70b-4bed-ad5d-91de8d726edc None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:40.953932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893453b-b70b-4bed-ad5d-91de8d726edc None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:40.954675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893453b-b70b-4bed-ad5d-91de8d726edc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:40.954982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893453b-b70b-4bed-ad5d-91de8d726edc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:40.956763 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893453b-b70b-4bed-ad5d-91de8d726edc None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:40.969539 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 473/938] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:40.970352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893453b-b70b-4bed-ad5d-91de8d726edc None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:40.993006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167575f3-2d42-48d0-93be-8775791ea514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:40.993684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167575f3-2d42-48d0-93be-8775791ea514 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:40.993969 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167575f3-2d42-48d0-93be-8775791ea514 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:40.994249 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167575f3-2d42-48d0-93be-8775791ea514 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:40.995480 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167575f3-2d42-48d0-93be-8775791ea514 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:40.996881 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167575f3-2d42-48d0-93be-8775791ea514 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:41.019615 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 466/939] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Jun 16 18:46:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:46:41.020265 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167575f3-2d42-48d0-93be-8775791ea514 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:41.126914 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893453b-b70b-4bed-ad5d-91de8d726edc None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:41.154223 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167575f3-2d42-48d0-93be-8775791ea514 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:41.178305 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b683fad-211a-4d57-9a5e-27855613044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:41.179159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b683fad-211a-4d57-9a5e-27855613044b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:41.179549 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b683fad-211a-4d57-9a5e-27855613044b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:41.179917 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b683fad-211a-4d57-9a5e-27855613044b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:41.180866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b683fad-211a-4d57-9a5e-27855613044b None admin] RBAC: Adding query filter params (interface=public, service_id=4bc7235fb3b54864b6d5dd143160a2c4, region_id=RegionOne) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:41.181274 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b683fad-211a-4d57-9a5e-27855613044b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:41.182926 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b683fad-211a-4d57-9a5e-27855613044b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:41.190143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 474/940] 10.222.0.70 () {56 vars in 1251 bytes} [Thu Jun 16 18:46:40 2022] GET /identity/v3/endpoints?service_id=4bc7235fb3b54864b6d5dd143160a2c4&interface=public®ion_id=RegionOne => generated 194 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:41.199763 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b683fad-211a-4d57-9a5e-27855613044b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:41.200782 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90826d-9bbe-42fb-b474-5bad05f95f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:41.201596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90826d-9bbe-42fb-b474-5bad05f95f33 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:41.201879 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90826d-9bbe-42fb-b474-5bad05f95f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:41.202152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90826d-9bbe-42fb-b474-5bad05f95f33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:41.206862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:41.213287 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90826d-9bbe-42fb-b474-5bad05f95f33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:41.215986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90826d-9bbe-42fb-b474-5bad05f95f33 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:41.235136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-5f90826d-9bbe-42fb-b474-5bad05f95f33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:41.235136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:41.235136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:41.235136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:41.235136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:41.235136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:41.235136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:41.235136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:41.235136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:41.235136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:41.235136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:41.235136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:41.235136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:41.235136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:41.235136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:41.236088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:41.236088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:41.236088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:41.236088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:41.236088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:41.236088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:41.236088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:41.236088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:41.236088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:41.236088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:41.236088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:41.236088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:41.236088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:41.236088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:41.236088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:41.236088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:41.236088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:41.236986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:41.236986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:41.236986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:41.236986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:41.236986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:41.236986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:41.236986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:41.236986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:41.239681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 467/941] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:41.246309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90826d-9bbe-42fb-b474-5bad05f95f33 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:41.270192 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:41.338152 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b683fad-211a-4d57-9a5e-27855613044b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:41.377652 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b46b04-9a01-402d-b81a-9c152a14da78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:41.378748 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b46b04-9a01-402d-b81a-9c152a14da78 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:41.379247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b46b04-9a01-402d-b81a-9c152a14da78 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:41.379712 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b46b04-9a01-402d-b81a-9c152a14da78 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:41.381343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b46b04-9a01-402d-b81a-9c152a14da78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:41.382941 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b46b04-9a01-402d-b81a-9c152a14da78 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:41.390289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 475/942] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Jun 16 18:46:41 2022] GET /identity/v3/services => generated 1497 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:46:41.401175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90826d-9bbe-42fb-b474-5bad05f95f33 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:41.443307 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599bc91-b6ce-4553-98e7-26a4d2a31cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:41.444159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599bc91-b6ce-4553-98e7-26a4d2a31cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:41.444447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599bc91-b6ce-4553-98e7-26a4d2a31cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:41.444727 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599bc91-b6ce-4553-98e7-26a4d2a31cd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:41.445554 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599bc91-b6ce-4553-98e7-26a4d2a31cd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:41.445866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599bc91-b6ce-4553-98e7-26a4d2a31cd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:41.447390 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599bc91-b6ce-4553-98e7-26a4d2a31cd4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:41.462251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 468/943] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:41.468315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b46b04-9a01-402d-b81a-9c152a14da78 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:41.589648 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b46b04-9a01-402d-b81a-9c152a14da78 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:41.610938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:41.630468 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683ee49-d1af-4cfe-8133-382328369423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:41.631824 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683ee49-d1af-4cfe-8133-382328369423 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:41.632312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683ee49-d1af-4cfe-8133-382328369423 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:41.632800 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683ee49-d1af-4cfe-8133-382328369423 None admin] PATH_INFO: `/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/02c826fc0baa480393df4c884c9e35dc/roles/ed8734ab73344d1983b1cdaa7fa3238a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:41.635365 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683ee49-d1af-4cfe-8133-382328369423 None admin] RBAC: Authorizing `identity:create_grant(project_id=34185125e75d4aad8ad5382ab22ee8a2, user_id=02c826fc0baa480393df4c884c9e35dc, role_id=ed8734ab73344d1983b1cdaa7fa3238a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:41.637032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683ee49-d1af-4cfe-8133-382328369423 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:41.653609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 476/944] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:46:41 2022] PUT /identity/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/02c826fc0baa480393df4c884c9e35dc/roles/ed8734ab73344d1983b1cdaa7fa3238a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:46:42.085549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88554ce-1f57-4dd2-b89a-edf1e9a49b55 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:42.086088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88554ce-1f57-4dd2-b89a-edf1e9a49b55 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:42.086088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88554ce-1f57-4dd2-b89a-edf1e9a49b55 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:42.087747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 469/945] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:46:42.092950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7ab2b-b8df-4b02-97f4-31c823a4f49b None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:42.093239 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7ab2b-b8df-4b02-97f4-31c823a4f49b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:42.093518 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7ab2b-b8df-4b02-97f4-31c823a4f49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:42.110740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-60c7ab2b-b8df-4b02-97f4-31c823a4f49b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:42.111923 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-60c7ab2b-b8df-4b02-97f4-31c823a4f49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:42.183824 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 477/946] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:42.207508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88554ce-1f57-4dd2-b89a-edf1e9a49b55 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:42.281018 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:42.328228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88554ce-1f57-4dd2-b89a-edf1e9a49b55 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:42.363242 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412db148-e528-42d6-a36e-ee27f2114b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:42.364275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412db148-e528-42d6-a36e-ee27f2114b0a None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:42.364791 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412db148-e528-42d6-a36e-ee27f2114b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:42.365310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412db148-e528-42d6-a36e-ee27f2114b0a None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:42.366232 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412db148-e528-42d6-a36e-ee27f2114b0a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:42.367887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412db148-e528-42d6-a36e-ee27f2114b0a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:42.369457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: WARNING keystone.server.flask.common [None req-412db148-e528-42d6-a36e-ee27f2114b0a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 18:46:42.396873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.sql.core [None req-412db148-e528-42d6-a36e-ee27f2114b0a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 18:46:42.396873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 18:46:42.396873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [parameters: {'id': '7b6d4e170ef44b258b4398710d53ffc2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 18:46:42.396873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80284) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 18:46:42.417960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-412db148-e528-42d6-a36e-ee27f2114b0a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7b6d4e170ef44b258b4398710d53ffc2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7b6d4e170ef44b258b4398710d53ffc2. Jun 16 18:46:42.417960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:42.417960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:46:42.417960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:46:42.417960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:46:42.417960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:46:42.417960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:46:42.417960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:46:42.417960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:46:42.417960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:46:42.417960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:46:42.417960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:46:42.417960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result.read() Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 16 18:46:42.418852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 16 18:46:42.419820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:46:42.419820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:46:42.419820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 18:46:42.419820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application yield resource Jun 16 18:46:42.419820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:46:42.419820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:46:42.419820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 18:46:42.419820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 18:46:42.419820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 18:46:42.419820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 18:46:42.419820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 18:46:42.419820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise self.value Jun 16 18:46:42.419820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 18:46:42.419820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 18:46:42.419820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 18:46:42.419820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application session.commit() Jun 16 18:46:42.419820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 18:46:42.420719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 18:46:42.420719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 18:46:42.420719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 18:46:42.420719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 18:46:42.420719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.session.flush() Jun 16 18:46:42.420719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 18:46:42.420719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 18:46:42.420719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 18:46:42.420719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 18:46:42.420719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 18:46:42.420719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application compat.raise_( Jun 16 18:46:42.420719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:46:42.420719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception Jun 16 18:46:42.420719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 18:46:42.420719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 18:46:42.420719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 18:46:42.420719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 18:46:42.421640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 18:46:42.421640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 18:46:42.421640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 18:46:42.421640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 18:46:42.421640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 18:46:42.421640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 18:46:42.421640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 18:46:42.421640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 18:46:42.421640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 18:46:42.421640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 18:46:42.421640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 18:46:42.421640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 18:46:42.421640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 18:46:42.421640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 18:46:42.421640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 18:46:42.422434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 18:46:42.422434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:46:42.422434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception Jun 16 18:46:42.422434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:46:42.422434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:46:42.422434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:46:42.422434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:46:42.422434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:46:42.422434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:46:42.422434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:46:42.422434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:46:42.422434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:46:42.422434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:46:42.422434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:46:42.422434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result.read() Jun 16 18:46:42.422434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:46:42.422434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:46:42.423341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:46:42.423341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:46:42.423341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:46:42.423341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:46:42.423341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:46:42.423341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:46:42.423341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 18:46:42.423341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 18:46:42.423341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [parameters: {'id': '7b6d4e170ef44b258b4398710d53ffc2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 18:46:42.423341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 18:46:42.423341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:42.423341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:42.423341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:42.423341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:42.423341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 16 18:46:42.424117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 16 18:46:42.424117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 18:46:42.424117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 18:46:42.424117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 16 18:46:42.424117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:42.424117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:42.424117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:42.424117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:42.424117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:42.424117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:42.424117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:42.424117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:42.424117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:42.424117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:42.424117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:42.424117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:42.424117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:42.426716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:42.426716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 16 18:46:42.426716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 16 18:46:42.426716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:42.426716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:42.426716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 16 18:46:42.426716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 16 18:46:42.426716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7b6d4e170ef44b258b4398710d53ffc2. Jun 16 18:46:42.426716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:42.426716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 470/947] 10.222.0.70 () {60 vars in 1140 bytes} [Thu Jun 16 18:46:42 2022] POST /identity/v3/projects => generated 262 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:42.428155 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7ab2b-b8df-4b02-97f4-31c823a4f49b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:42.530893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7ab2b-b8df-4b02-97f4-31c823a4f49b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:42.532890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:42.568362 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c9c49-5c2c-4b98-984f-0cde77655c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:42.569150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c9c49-5c2c-4b98-984f-0cde77655c30 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:42.569430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c9c49-5c2c-4b98-984f-0cde77655c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:42.569705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c9c49-5c2c-4b98-984f-0cde77655c30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:42.576438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c9c49-5c2c-4b98-984f-0cde77655c30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:42.577905 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c9c49-5c2c-4b98-984f-0cde77655c30 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:42.592671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-2b9c9c49-5c2c-4b98-984f-0cde77655c30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:42.592671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:42.592671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:42.592671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:42.592671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:42.592671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:42.592671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:42.592671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:42.592671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:42.592671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:42.592671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:42.592671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:42.592671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:42.592671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:42.592671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:42.593798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:42.593798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:42.593798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:42.593798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:42.593798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:42.593798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:42.593798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:42.593798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:42.593798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:42.593798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:42.593798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:42.593798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:42.593798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:42.593798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:42.593798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:42.593798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:42.593798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:42.594890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:42.594890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:42.594890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:42.594890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:42.594890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:42.594890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:42.594890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:42.594890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:42.594890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 478/948] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:42.601840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412db148-e528-42d6-a36e-ee27f2114b0a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:42.713324 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412db148-e528-42d6-a36e-ee27f2114b0a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:42.726910 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:42.749244 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a05e60-1496-4ae1-bd76-5b52ba9c6c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:42.750229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a05e60-1496-4ae1-bd76-5b52ba9c6c41 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:42.751045 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a05e60-1496-4ae1-bd76-5b52ba9c6c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:42.751380 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a05e60-1496-4ae1-bd76-5b52ba9c6c41 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:42.752262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a05e60-1496-4ae1-bd76-5b52ba9c6c41 None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:42.753013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a05e60-1496-4ae1-bd76-5b52ba9c6c41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:42.755071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a05e60-1496-4ae1-bd76-5b52ba9c6c41 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:42.780371 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 471/949] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Jun 16 18:46:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:43.311377 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8bbae-f263-4a39-af80-f88d067100a3 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:43.311870 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8bbae-f263-4a39-af80-f88d067100a3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:43.311870 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8bbae-f263-4a39-af80-f88d067100a3 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:43.313514 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 479/950] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:43.319847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071aa84d-cbda-4f2b-abc0-5c20e2722cbf None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:43.320308 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071aa84d-cbda-4f2b-abc0-5c20e2722cbf None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:43.320494 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071aa84d-cbda-4f2b-abc0-5c20e2722cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:43.338124 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-071aa84d-cbda-4f2b-abc0-5c20e2722cbf None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:43.339688 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-071aa84d-cbda-4f2b-abc0-5c20e2722cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:43.414869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 472/951] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:43.436884 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8bbae-f263-4a39-af80-f88d067100a3 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:43.549197 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8bbae-f263-4a39-af80-f88d067100a3 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:43.596709 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61a084-ea74-48aa-b124-f6ab65d52667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:43.597623 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61a084-ea74-48aa-b124-f6ab65d52667 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:43.598041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61a084-ea74-48aa-b124-f6ab65d52667 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:43.598447 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61a084-ea74-48aa-b124-f6ab65d52667 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:43.599533 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61a084-ea74-48aa-b124-f6ab65d52667 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:43.601743 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61a084-ea74-48aa-b124-f6ab65d52667 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:43.619242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-0a61a084-ea74-48aa-b124-f6ab65d52667 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:46:43.619242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:43.619242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:43.619242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:43.619242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:43.619242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:43.619242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:43.619242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:43.619242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:46:43.619242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:43.619242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:43.619242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:43.619242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:43.619242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:43.619242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:43.619242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:43.619242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:43.620340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:43.620340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:43.620340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:43.620340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:43.620340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:43.620340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:43.620340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:43.620340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:43.620340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:43.620340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:43.620340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:43.620340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:43.620340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:43.620340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:43.620340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:43.620340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:43.621185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:43.621185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:43.621185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:43.621185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:43.621185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:43.621185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:43.621185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:43.621185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:43.621185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:43.621185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:43.621185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:43.621185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:43.621185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:46:43.621185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:43.621185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 480/952] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:43.627013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071aa84d-cbda-4f2b-abc0-5c20e2722cbf None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:43.715273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5321b-5d11-4817-8741-e391fc600635 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:43.715605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5321b-5d11-4817-8741-e391fc600635 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:43.716268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5321b-5d11-4817-8741-e391fc600635 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:43.718205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 481/953] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:43.725231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10ce54-e9d5-427b-bd61-7424062f4f44 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:43.725512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10ce54-e9d5-427b-bd61-7424062f4f44 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:43.725788 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10ce54-e9d5-427b-bd61-7424062f4f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:43.747746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071aa84d-cbda-4f2b-abc0-5c20e2722cbf None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:43.748290 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ed10ce54-e9d5-427b-bd61-7424062f4f44 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:43.749480 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:43.750060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ed10ce54-e9d5-427b-bd61-7424062f4f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:43.776767 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:43.796964 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11273654-614c-4131-b7eb-724c9f2115c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:43.798419 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11273654-614c-4131-b7eb-724c9f2115c0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:43.798888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11273654-614c-4131-b7eb-724c9f2115c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:43.799306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11273654-614c-4131-b7eb-724c9f2115c0 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:43.800156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11273654-614c-4131-b7eb-724c9f2115c0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:43.801044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11273654-614c-4131-b7eb-724c9f2115c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:43.803422 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11273654-614c-4131-b7eb-724c9f2115c0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:43.811533 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 473/954] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Jun 16 18:46:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:43.818098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11273654-614c-4131-b7eb-724c9f2115c0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:43.842824 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 482/955] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:46:43.868106 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10ce54-e9d5-427b-bd61-7424062f4f44 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:43.938498 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11273654-614c-4131-b7eb-724c9f2115c0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:43.984876 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10ce54-e9d5-427b-bd61-7424062f4f44 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:43.986446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47480b-7034-44ff-9882-7e0cc1cb04f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:43.987152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47480b-7034-44ff-9882-7e0cc1cb04f5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:43.987435 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47480b-7034-44ff-9882-7e0cc1cb04f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:43.987710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47480b-7034-44ff-9882-7e0cc1cb04f5 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:43.988428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47480b-7034-44ff-9882-7e0cc1cb04f5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:43.988735 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47480b-7034-44ff-9882-7e0cc1cb04f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:43.990929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47480b-7034-44ff-9882-7e0cc1cb04f5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:43.995565 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 474/956] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Jun 16 18:46:43 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:43.996870 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fbf26-600a-434c-b440-3332cce299a7 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:43.997160 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fbf26-600a-434c-b440-3332cce299a7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:43.997437 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fbf26-600a-434c-b440-3332cce299a7 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:43.998909 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 475/957] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:44.004559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96603514-3675-469d-b6fa-f8c0210311c8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:44.004911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96603514-3675-469d-b6fa-f8c0210311c8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:44.005335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96603514-3675-469d-b6fa-f8c0210311c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:44.024657 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-96603514-3675-469d-b6fa-f8c0210311c8 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:44.025877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-96603514-3675-469d-b6fa-f8c0210311c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:44.045077 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9dfbbf-f141-4f22-9a22-d362f6fae4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:44.046161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dfbbf-f141-4f22-9a22-d362f6fae4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:44.046441 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dfbbf-f141-4f22-9a22-d362f6fae4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:44.046748 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dfbbf-f141-4f22-9a22-d362f6fae4c7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:44.047417 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9dfbbf-f141-4f22-9a22-d362f6fae4c7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:44.048823 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9dfbbf-f141-4f22-9a22-d362f6fae4c7 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:44.060593 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:44.069972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-5f9dfbbf-f141-4f22-9a22-d362f6fae4c7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:46:44.069972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:44.069972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:44.069972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:44.069972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:44.069972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:44.069972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:44.069972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:44.069972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:46:44.069972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:44.069972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:44.069972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:44.069972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:44.069972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:44.069972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:44.069972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:44.069972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:44.071145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:44.071145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:44.071145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:44.071145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:44.071145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:44.071145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:44.071145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:44.071145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:44.071145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:44.071145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:44.071145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:44.071145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:44.071145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:44.071145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:44.071145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:44.071145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:44.072033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:44.072033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:44.072033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:44.072033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:44.072033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:44.072033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:44.072033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:44.072033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:44.072033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:44.072033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:44.072033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:44.072033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:44.072033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:46:44.072033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:44.072768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 483/958] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Jun 16 18:46:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:44.078687 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9dfbbf-f141-4f22-9a22-d362f6fae4c7 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:44.107865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:44.119628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 476/959] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:46:44.144171 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96603514-3675-469d-b6fa-f8c0210311c8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:44.207106 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9dfbbf-f141-4f22-9a22-d362f6fae4c7 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:44.252020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8f101-d322-4b20-b048-945449a0eb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:44.252951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8f101-d322-4b20-b048-945449a0eb96 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:44.253236 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8f101-d322-4b20-b048-945449a0eb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:44.253763 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8f101-d322-4b20-b048-945449a0eb96 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:44.254974 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8f101-d322-4b20-b048-945449a0eb96 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:44.255291 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8f101-d322-4b20-b048-945449a0eb96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:44.257206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8f101-d322-4b20-b048-945449a0eb96 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:44.261156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96603514-3675-469d-b6fa-f8c0210311c8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:44.264277 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 484/960] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Jun 16 18:46:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:44.269166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8f101-d322-4b20-b048-945449a0eb96 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:44.305494 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2937e-8ce4-4a8e-a7a5-c589fa3e5da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:44.306384 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2937e-8ce4-4a8e-a7a5-c589fa3e5da8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:44.307023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2937e-8ce4-4a8e-a7a5-c589fa3e5da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:44.307143 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2937e-8ce4-4a8e-a7a5-c589fa3e5da8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:44.318199 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2937e-8ce4-4a8e-a7a5-c589fa3e5da8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:44.326980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-92d2937e-8ce4-4a8e-a7a5-c589fa3e5da8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:46:44.326980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:44.326980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:44.326980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:44.326980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:44.326980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:44.326980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:44.326980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:44.326980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:44.326980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:44.326980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:44.326980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:44.326980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:44.326980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:44.326980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:44.326980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:44.328568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:44.328568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:44.328568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:44.328568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:44.328568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:44.328568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:44.328568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:44.328568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:44.328568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:44.328568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:44.328568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:44.328568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:44.328568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:44.328568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:44.328568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:44.328568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:44.329568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:44.329568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:44.329568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:44.329568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:44.329568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:44.329568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:44.329568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:44.329568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:44.329568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:44.329568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:44.329568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:46:44.329568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:44.333734 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 477/961] 10.222.0.70 () {56 vars in 1120 bytes} [Thu Jun 16 18:46:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:44.335205 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2937e-8ce4-4a8e-a7a5-c589fa3e5da8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:44.359567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:44.372476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:44.391488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8f101-d322-4b20-b048-945449a0eb96 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:44.429893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c30253-10d2-4232-9c90-f1964318fcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:44.430847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c30253-10d2-4232-9c90-f1964318fcf4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:44.431279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c30253-10d2-4232-9c90-f1964318fcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:44.431745 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c30253-10d2-4232-9c90-f1964318fcf4 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:44.432815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c30253-10d2-4232-9c90-f1964318fcf4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:44.433301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c30253-10d2-4232-9c90-f1964318fcf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:44.434843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c30253-10d2-4232-9c90-f1964318fcf4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:44.445352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 485/962] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Jun 16 18:46:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:44.446523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c30253-10d2-4232-9c90-f1964318fcf4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:44.449007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2937e-8ce4-4a8e-a7a5-c589fa3e5da8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:44.488785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2efe7-70d8-41c1-afae-59da4da00c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:44.490424 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2efe7-70d8-41c1-afae-59da4da00c5f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:44.490927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2efe7-70d8-41c1-afae-59da4da00c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:44.491374 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2efe7-70d8-41c1-afae-59da4da00c5f None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:44.492826 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2efe7-70d8-41c1-afae-59da4da00c5f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:44.493291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2efe7-70d8-41c1-afae-59da4da00c5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:44.495772 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2efe7-70d8-41c1-afae-59da4da00c5f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:44.511307 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 478/963] 10.222.0.70 () {56 vars in 1115 bytes} [Thu Jun 16 18:46:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:44.513032 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2efe7-70d8-41c1-afae-59da4da00c5f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:44.561405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c30253-10d2-4232-9c90-f1964318fcf4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:44.600497 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bca78-41da-466f-b2e8-1f6fd1b2c4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:44.601406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bca78-41da-466f-b2e8-1f6fd1b2c4b2 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:44.601842 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bca78-41da-466f-b2e8-1f6fd1b2c4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:44.602394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bca78-41da-466f-b2e8-1f6fd1b2c4b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:44.603373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:44.604464 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bca78-41da-466f-b2e8-1f6fd1b2c4b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:44.606830 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bca78-41da-466f-b2e8-1f6fd1b2c4b2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:44.623629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2efe7-70d8-41c1-afae-59da4da00c5f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:44.642919 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 486/964] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Jun 16 18:46:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:46:44.679789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcee6a9-0531-4fa6-8d60-2e015b9c80b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:44.680687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcee6a9-0531-4fa6-8d60-2e015b9c80b6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:44.680977 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcee6a9-0531-4fa6-8d60-2e015b9c80b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:44.681272 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcee6a9-0531-4fa6-8d60-2e015b9c80b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:44.695204 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcee6a9-0531-4fa6-8d60-2e015b9c80b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:44.696748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcee6a9-0531-4fa6-8d60-2e015b9c80b6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:44.716748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-4bcee6a9-0531-4fa6-8d60-2e015b9c80b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:44.716748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:44.716748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:44.716748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:44.716748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:44.716748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:44.716748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:44.716748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:44.716748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:44.716748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:44.716748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:44.716748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:44.716748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:44.716748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:44.716748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:44.717732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:44.717732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:44.717732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:44.717732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:44.717732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:44.717732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:44.717732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:44.717732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:44.717732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:44.717732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:44.717732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:44.717732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:44.717732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:44.717732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:44.717732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:44.717732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:44.717732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:44.718675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:44.718675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:44.718675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:44.718675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:44.722738 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 479/965] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:44.724675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bca78-41da-466f-b2e8-1f6fd1b2c4b2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:44.729845 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:44.849654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bca78-41da-466f-b2e8-1f6fd1b2c4b2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:44.886839 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95afda0-c22e-4fac-bd37-da4a12ad64e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:44.887435 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95afda0-c22e-4fac-bd37-da4a12ad64e6 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:44.887719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95afda0-c22e-4fac-bd37-da4a12ad64e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:44.887996 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95afda0-c22e-4fac-bd37-da4a12ad64e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:44.888770 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95afda0-c22e-4fac-bd37-da4a12ad64e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:44.889084 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95afda0-c22e-4fac-bd37-da4a12ad64e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:44.890456 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95afda0-c22e-4fac-bd37-da4a12ad64e6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:44.904836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 487/966] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:44.910697 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcee6a9-0531-4fa6-8d60-2e015b9c80b6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:45.027655 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcee6a9-0531-4fa6-8d60-2e015b9c80b6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:45.061138 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:45.068241 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd74fb-39b6-40f4-9b0c-e4b232aa851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:45.069066 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd74fb-39b6-40f4-9b0c-e4b232aa851f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:45.069353 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd74fb-39b6-40f4-9b0c-e4b232aa851f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:45.069632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd74fb-39b6-40f4-9b0c-e4b232aa851f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:45.078627 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fd74fb-39b6-40f4-9b0c-e4b232aa851f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:45.080103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fd74fb-39b6-40f4-9b0c-e4b232aa851f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:45.086717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f02cb-fff4-4c28-9e6b-1775d80bf7f2 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:45.087010 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f02cb-fff4-4c28-9e6b-1775d80bf7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:45.088557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f02cb-fff4-4c28-9e6b-1775d80bf7f2 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:45.090177 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 488/967] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:46:45.095229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204ca97-ea3d-42e1-a587-5a8a23848ffe None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:45.095517 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204ca97-ea3d-42e1-a587-5a8a23848ffe None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:45.095793 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204ca97-ea3d-42e1-a587-5a8a23848ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:45.096543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-82fd74fb-39b6-40f4-9b0c-e4b232aa851f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:46:45.096543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:45.096543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:45.096543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:45.096543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:45.096543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:45.096543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:45.096543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:45.096543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:45.096543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:45.096543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:45.096543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:45.096543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:45.096543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:45.096543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:45.096543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:45.097549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:45.099313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:45.099313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:45.099313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:45.099313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:45.099313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:45.099313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:45.099313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:45.099313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:45.099313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:46:45.099313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:45.099313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 480/968] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Jun 16 18:46:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:45.103643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd74fb-39b6-40f4-9b0c-e4b232aa851f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:45.113697 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5204ca97-ea3d-42e1-a587-5a8a23848ffe None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:45.115685 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5204ca97-ea3d-42e1-a587-5a8a23848ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:45.182678 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:45.193776 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 489/969] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:45.218062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd74fb-39b6-40f4-9b0c-e4b232aa851f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:45.221319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204ca97-ea3d-42e1-a587-5a8a23848ffe None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:45.262151 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff79868-84fc-411d-9fe8-eb320ed38147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:45.262758 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff79868-84fc-411d-9fe8-eb320ed38147 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:45.263048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff79868-84fc-411d-9fe8-eb320ed38147 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:45.263324 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff79868-84fc-411d-9fe8-eb320ed38147 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:45.264184 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff79868-84fc-411d-9fe8-eb320ed38147 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:45.264496 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff79868-84fc-411d-9fe8-eb320ed38147 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:45.265937 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff79868-84fc-411d-9fe8-eb320ed38147 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:45.296254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 481/970] 10.222.0.70 () {56 vars in 1149 bytes} [Thu Jun 16 18:46:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:45.301962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff79868-84fc-411d-9fe8-eb320ed38147 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:45.350985 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204ca97-ea3d-42e1-a587-5a8a23848ffe None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:45.355553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:45.397786 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45081251-106e-44e3-98a0-9bcc1083cefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:45.399177 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45081251-106e-44e3-98a0-9bcc1083cefa None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:45.399663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45081251-106e-44e3-98a0-9bcc1083cefa None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:45.400138 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45081251-106e-44e3-98a0-9bcc1083cefa None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:45.401060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45081251-106e-44e3-98a0-9bcc1083cefa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:45.402749 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45081251-106e-44e3-98a0-9bcc1083cefa None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:45.404271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: WARNING keystone.server.flask.common [None req-45081251-106e-44e3-98a0-9bcc1083cefa None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 18:46:45.417356 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff79868-84fc-411d-9fe8-eb320ed38147 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:45.418862 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.sql.core [None req-45081251-106e-44e3-98a0-9bcc1083cefa None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 18:46:45.418862 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 18:46:45.418862 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [parameters: {'user_id': 'bebdfc07a2fd4f2fb109c0b477edbcc0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 18:46:45.418862 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80283) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 18:46:45.427218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-45081251-106e-44e3-98a0-9bcc1083cefa None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 18:46:45.427218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:45.427218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:46:45.427218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:46:45.427218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:46:45.427218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:46:45.427218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:46:45.427218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:46:45.427218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:46:45.427218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:46:45.427218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:46:45.427218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:46:45.427218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:46:45.427218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application result.read() Jun 16 18:46:45.427218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:46:45.428100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:46:45.429062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 18:46:45.429062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application yield resource Jun 16 18:46:45.429062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:46:45.429062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:46:45.429062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 18:46:45.429062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 18:46:45.429062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 18:46:45.429062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 18:46:45.429062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 18:46:45.429062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise self.value Jun 16 18:46:45.429062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 18:46:45.429062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 18:46:45.429062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 18:46:45.429062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application session.commit() Jun 16 18:46:45.429062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 18:46:45.429062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 18:46:45.429062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 18:46:45.430006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 18:46:45.430006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 18:46:45.430006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self.session.flush() Jun 16 18:46:45.430006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 18:46:45.430006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 18:46:45.430006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 18:46:45.430006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 18:46:45.430006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 18:46:45.430006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application compat.raise_( Jun 16 18:46:45.430006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:46:45.430006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception Jun 16 18:46:45.430006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 18:46:45.430006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 18:46:45.430006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 18:46:45.430006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 18:46:45.430006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 18:46:45.430006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 18:46:45.431651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 18:46:45.431651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 18:46:45.431651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 16 18:46:45.431651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 16 18:46:45.431651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 18:46:45.431651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 18:46:45.431651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 18:46:45.431651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 18:46:45.431651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 18:46:45.431651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 18:46:45.431651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 18:46:45.431651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 18:46:45.431651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 18:46:45.431651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 18:46:45.431651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:46:45.431651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception Jun 16 18:46:45.432524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:46:45.432524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:46:45.432524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:46:45.432524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:46:45.432524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:46:45.432524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:46:45.432524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:46:45.432524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:46:45.432524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:46:45.432524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:46:45.432524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:46:45.432524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application result.read() Jun 16 18:46:45.432524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:46:45.432524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:46:45.432524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:46:45.432524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:46:45.433465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:46:45.433465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:46:45.433465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:46:45.433465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:46:45.433465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 18:46:45.433465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 18:46:45.433465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [parameters: {'user_id': 'bebdfc07a2fd4f2fb109c0b477edbcc0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 18:46:45.433465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 18:46:45.433465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:45.433465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:45.433465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:45.433465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:45.433465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:45.433465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:45.433465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:45.434356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:45.434356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:45.434356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:45.434356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:45.434356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:45.434356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:45.434356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:45.434356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 16 18:46:45.434356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 16 18:46:45.434356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:45.434356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:45.434356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:45.434356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:45.434356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:45.434356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:45.434356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 16 18:46:45.434356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 16 18:46:45.436978 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 16 18:46:45.436978 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 16 18:46:45.436978 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 18:46:45.436978 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 18:46:45.436978 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 18:46:45.436978 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:45.439588 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 490/971] 10.222.0.70 () {60 vars in 1131 bytes} [Thu Jun 16 18:46:45 2022] POST /identity/v3/users => generated 160 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:45.448126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45081251-106e-44e3-98a0-9bcc1083cefa None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:45.466495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4aadd-c577-419b-8de4-9a268fff3b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:45.468195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4aadd-c577-419b-8de4-9a268fff3b8e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:45.468809 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4aadd-c577-419b-8de4-9a268fff3b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:45.469322 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4aadd-c577-419b-8de4-9a268fff3b8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:45.512522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:45.564960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4aadd-c577-419b-8de4-9a268fff3b8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:45.566837 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4aadd-c577-419b-8de4-9a268fff3b8e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:45.572409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45081251-106e-44e3-98a0-9bcc1083cefa None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:45.609035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe80d51-3ff1-4759-bd8b-d8d512cf6550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:45.610013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe80d51-3ff1-4759-bd8b-d8d512cf6550 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:45.610202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe80d51-3ff1-4759-bd8b-d8d512cf6550 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:45.610477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe80d51-3ff1-4759-bd8b-d8d512cf6550 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:45.618248 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:45.619741 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe80d51-3ff1-4759-bd8b-d8d512cf6550 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:45.621371 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe80d51-3ff1-4759-bd8b-d8d512cf6550 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:45.638583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-afe80d51-3ff1-4759-bd8b-d8d512cf6550 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:46:45.638583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:45.638583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:45.638583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:45.638583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:45.638583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:45.638583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:45.638583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:45.638583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:45.638583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:45.638583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:45.638583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:45.638583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:45.638583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:45.638583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:45.638583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:45.639576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:45.640515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:45.640515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:45.640515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:45.640515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:45.640515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:45.640515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:45.640515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:45.640515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:45.640515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:46:45.640515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:45.644857 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 491/972] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:46:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:45.645906 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe80d51-3ff1-4759-bd8b-d8d512cf6550 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:45.686200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 482/973] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:46:45 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:46:45.687837 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4aadd-c577-419b-8de4-9a268fff3b8e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:45.758974 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe80d51-3ff1-4759-bd8b-d8d512cf6550 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:45.777304 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:45.794056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4aadd-c577-419b-8de4-9a268fff3b8e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:45.798696 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb2b5f-6972-468a-bed3-957267f955f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:45.799558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb2b5f-6972-468a-bed3-957267f955f2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:45.799991 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb2b5f-6972-468a-bed3-957267f955f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:45.800409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb2b5f-6972-468a-bed3-957267f955f2 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:45.801366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb2b5f-6972-468a-bed3-957267f955f2 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:45.802351 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb2b5f-6972-468a-bed3-957267f955f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:45.804060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb2b5f-6972-468a-bed3-957267f955f2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:45.832626 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc635bfc-cd72-46e1-8af5-152d03be8cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:45.834639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc635bfc-cd72-46e1-8af5-152d03be8cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:45.835038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc635bfc-cd72-46e1-8af5-152d03be8cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:45.835418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc635bfc-cd72-46e1-8af5-152d03be8cc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:45.847470 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc635bfc-cd72-46e1-8af5-152d03be8cc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:45.854605 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc635bfc-cd72-46e1-8af5-152d03be8cc2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:45.855225 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 492/974] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Jun 16 18:46:45 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:45.860348 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:45.871931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-fc635bfc-cd72-46e1-8af5-152d03be8cc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:45.871931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:45.871931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:45.871931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:45.871931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:45.871931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:45.871931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:45.871931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:45.871931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:45.871931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:45.871931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:45.871931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:45.871931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:45.871931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:45.871931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:45.873071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:45.873071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:45.873071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:45.873071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:45.873071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:45.873071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:45.873071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:45.873071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:45.873071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:45.873071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:45.873071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:45.873071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:45.873071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:45.873071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:45.873071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:45.873071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:45.873071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:45.874248 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:45.874248 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:45.874248 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:45.874248 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:45.880594 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 483/975] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:45.882213 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb2b5f-6972-468a-bed3-957267f955f2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:46.009475 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb2b5f-6972-468a-bed3-957267f955f2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:46.053154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d413417-8817-40a3-b6df-ac344a870120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:46.054076 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d413417-8817-40a3-b6df-ac344a870120 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:46.054506 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d413417-8817-40a3-b6df-ac344a870120 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:46.054981 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d413417-8817-40a3-b6df-ac344a870120 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:46.055853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d413417-8817-40a3-b6df-ac344a870120 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:46.056304 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d413417-8817-40a3-b6df-ac344a870120 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:46.057811 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d413417-8817-40a3-b6df-ac344a870120 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:46.071339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 493/976] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:46.077104 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc635bfc-cd72-46e1-8af5-152d03be8cc2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:46.190998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc635bfc-cd72-46e1-8af5-152d03be8cc2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:46.227394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7391a0f-1953-4449-980e-f2a488701bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:46.228280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7391a0f-1953-4449-980e-f2a488701bbd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:46.228676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7391a0f-1953-4449-980e-f2a488701bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:46.229060 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7391a0f-1953-4449-980e-f2a488701bbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:46.236215 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7391a0f-1953-4449-980e-f2a488701bbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:46.237833 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7391a0f-1953-4449-980e-f2a488701bbd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:46.253264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-a7391a0f-1953-4449-980e-f2a488701bbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:46.253264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:46.253264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:46.253264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:46.253264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:46.253264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:46.253264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:46.253264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:46.253264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:46.253264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:46.253264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:46.253264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:46.253264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:46.253264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:46.253264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:46.254443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:46.254443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:46.254443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:46.254443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:46.254443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:46.254443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:46.254443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:46.254443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:46.254443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:46.254443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:46.254443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:46.254443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:46.254443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:46.254443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:46.254443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:46.254443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:46.254443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:46.255908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:46.255908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:46.255908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:46.255908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:46.255908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:46.255908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:46.255908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:46.255908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:46.258651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 484/977] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:46.264404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d413417-8817-40a3-b6df-ac344a870120 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:46.302633 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:46.317983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579f4ed-27e0-4b54-8eea-4ae1d551df88 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:46.318484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579f4ed-27e0-4b54-8eea-4ae1d551df88 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:46.318939 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579f4ed-27e0-4b54-8eea-4ae1d551df88 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:46.320568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 485/978] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:46.325487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c2fa0-dde5-41c3-aeeb-914f57656578 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:46.325778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c2fa0-dde5-41c3-aeeb-914f57656578 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:46.326065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c2fa0-dde5-41c3-aeeb-914f57656578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:46.343475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d56c2fa0-dde5-41c3-aeeb-914f57656578 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:46.344656 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d56c2fa0-dde5-41c3-aeeb-914f57656578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:46.375274 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d413417-8817-40a3-b6df-ac344a870120 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:46.403844 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:46.415968 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e461f-e06b-4086-8aac-296cdadaa4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:46.416637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e461f-e06b-4086-8aac-296cdadaa4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:46.416920 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e461f-e06b-4086-8aac-296cdadaa4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:46.417197 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e461f-e06b-4086-8aac-296cdadaa4d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:46.417986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e461f-e06b-4086-8aac-296cdadaa4d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:46.418300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e461f-e06b-4086-8aac-296cdadaa4d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:46.419768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e461f-e06b-4086-8aac-296cdadaa4d5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:46.422702 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 486/979] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:46:46.438422 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 494/980] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:46.443837 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c2fa0-dde5-41c3-aeeb-914f57656578 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:46.444889 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e461f-e06b-4086-8aac-296cdadaa4d5 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:46.548097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:46.550957 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c2fa0-dde5-41c3-aeeb-914f57656578 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:46.556000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e461f-e06b-4086-8aac-296cdadaa4d5 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:46.595777 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa0b64-cd38-490d-9111-9c0a3841c358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:46.596451 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa0b64-cd38-490d-9111-9c0a3841c358 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:46.596736 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa0b64-cd38-490d-9111-9c0a3841c358 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:46.596918 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b9cb8-50ee-4e6f-b6d6-a624f1198fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:46.597026 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa0b64-cd38-490d-9111-9c0a3841c358 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:46.597505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b9cb8-50ee-4e6f-b6d6-a624f1198fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:46.597730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa0b64-cd38-490d-9111-9c0a3841c358 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:46.597825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b9cb8-50ee-4e6f-b6d6-a624f1198fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:46.598049 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b9cb8-50ee-4e6f-b6d6-a624f1198fa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:46.599641 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b9cb8-50ee-4e6f-b6d6-a624f1198fa1 None admin] RBAC: Adding query filter params (role.id=ed8734ab73344d1983b1cdaa7fa3238a, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=02c826fc0baa480393df4c884c9e35dc) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:46.599891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa0b64-cd38-490d-9111-9c0a3841c358 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:46.600174 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b9cb8-50ee-4e6f-b6d6-a624f1198fa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:46.601597 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b9cb8-50ee-4e6f-b6d6-a624f1198fa1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:46.609170 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 495/981] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:46:46 2022] GET /identity/v3/role_assignments?user.id=02c826fc0baa480393df4c884c9e35dc&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=ed8734ab73344d1983b1cdaa7fa3238a => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:46.620780 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 487/982] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Jun 16 18:46:46 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:46:46.906722 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133c60a-0981-4b8c-933d-f2e4a699f27a None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:46.906906 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133c60a-0981-4b8c-933d-f2e4a699f27a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:46.907189 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133c60a-0981-4b8c-933d-f2e4a699f27a None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:46.908547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 496/983] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:46.914117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dc90d-3ff8-4318-8426-70296fee5abd None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:46.914408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dc90d-3ff8-4318-8426-70296fee5abd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:46.914723 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dc90d-3ff8-4318-8426-70296fee5abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:46.933025 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-746dc90d-3ff8-4318-8426-70296fee5abd None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:46.934212 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-746dc90d-3ff8-4318-8426-70296fee5abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:46.994025 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:47.026274 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 488/984] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:47.048552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133c60a-0981-4b8c-933d-f2e4a699f27a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:47.152848 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133c60a-0981-4b8c-933d-f2e4a699f27a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:47.187614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ea92f-9483-4e41-9137-6977e5123062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:47.188415 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ea92f-9483-4e41-9137-6977e5123062 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:47.188691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ea92f-9483-4e41-9137-6977e5123062 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:47.188960 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ea92f-9483-4e41-9137-6977e5123062 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:47.189604 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ea92f-9483-4e41-9137-6977e5123062 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:47.191042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ea92f-9483-4e41-9137-6977e5123062 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:47.203358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-d14ea92f-9483-4e41-9137-6977e5123062 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:46:47.203358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:47.203358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:47.203358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:47.203358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:47.203358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:47.203358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:47.203358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:47.203358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:46:47.203358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:47.203358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:47.203358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:47.203358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:47.203358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:47.203358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:47.203358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:47.203358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:47.204372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:47.204372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:47.204372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:47.204372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:47.204372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:47.204372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:47.204372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:47.204372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:47.204372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:47.204372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:47.204372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:47.204372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:47.204372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:47.204372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:47.204372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:47.204372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:47.205348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:47.205348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:47.205348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:47.205348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:47.205348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:47.205348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:47.205348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:47.205348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:47.205348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:47.205348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:47.205348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:47.205348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:47.205348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:46:47.205348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:47.213483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 497/985] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Jun 16 18:46:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:47.213621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dc90d-3ff8-4318-8426-70296fee5abd None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:47.313275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dc90d-3ff8-4318-8426-70296fee5abd None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:47.347827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22021993-36d7-414a-867f-0cbcf5ce2f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:47.348597 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22021993-36d7-414a-867f-0cbcf5ce2f96 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:47.348872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22021993-36d7-414a-867f-0cbcf5ce2f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:47.349142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22021993-36d7-414a-867f-0cbcf5ce2f96 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:47.349843 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22021993-36d7-414a-867f-0cbcf5ce2f96 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:47.350165 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22021993-36d7-414a-867f-0cbcf5ce2f96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:47.351521 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22021993-36d7-414a-867f-0cbcf5ce2f96 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:47.362600 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 489/986] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Jun 16 18:46:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:47.362741 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ea92f-9483-4e41-9137-6977e5123062 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:47.389455 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:47.468470 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ea92f-9483-4e41-9137-6977e5123062 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:47.502385 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03a512-460a-4ea5-9914-50655ba657ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:47.503076 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03a512-460a-4ea5-9914-50655ba657ce None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:47.503357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03a512-460a-4ea5-9914-50655ba657ce None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:47.503628 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03a512-460a-4ea5-9914-50655ba657ce None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:47.504324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03a512-460a-4ea5-9914-50655ba657ce None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:47.504704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03a512-460a-4ea5-9914-50655ba657ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:47.506287 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03a512-460a-4ea5-9914-50655ba657ce None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:47.511314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 498/987] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Jun 16 18:46:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:47.516864 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22021993-36d7-414a-867f-0cbcf5ce2f96 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:47.559684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:47.618296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22021993-36d7-414a-867f-0cbcf5ce2f96 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:47.652254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab953b58-541d-42a3-b80b-4529c829f227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:47.652898 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab953b58-541d-42a3-b80b-4529c829f227 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:47.653178 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab953b58-541d-42a3-b80b-4529c829f227 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:47.653451 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab953b58-541d-42a3-b80b-4529c829f227 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:47.654233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab953b58-541d-42a3-b80b-4529c829f227 None admin] RBAC: Adding query filter params (interface=admin, service_id=4bc7235fb3b54864b6d5dd143160a2c4, region_id=RegionOne) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:47.654541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab953b58-541d-42a3-b80b-4529c829f227 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:47.655915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab953b58-541d-42a3-b80b-4529c829f227 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:47.661778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 490/988] 10.222.0.70 () {56 vars in 1249 bytes} [Thu Jun 16 18:46:47 2022] GET /identity/v3/endpoints?service_id=4bc7235fb3b54864b6d5dd143160a2c4&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:47.667431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03a512-460a-4ea5-9914-50655ba657ce None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:47.753889 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:47.772111 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03a512-460a-4ea5-9914-50655ba657ce None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:47.809494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db05067-dc69-475a-9ea4-bd1cb95b0160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:47.810156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db05067-dc69-475a-9ea4-bd1cb95b0160 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:47.810434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db05067-dc69-475a-9ea4-bd1cb95b0160 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:47.810750 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db05067-dc69-475a-9ea4-bd1cb95b0160 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:47.811418 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db05067-dc69-475a-9ea4-bd1cb95b0160 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:47.812790 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db05067-dc69-475a-9ea4-bd1cb95b0160 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:47.819697 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 499/989] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Jun 16 18:46:47 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:46:48.850548 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac0d85-3e4e-4ca1-9004-d27df39c8d23 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:48.851104 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac0d85-3e4e-4ca1-9004-d27df39c8d23 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:48.851183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac0d85-3e4e-4ca1-9004-d27df39c8d23 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:48.852649 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 491/990] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:48.857612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd993c6-7743-4d37-934c-f79e30827e4e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:48.858020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd993c6-7743-4d37-934c-f79e30827e4e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:48.858234 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd993c6-7743-4d37-934c-f79e30827e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:48.875545 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5cd993c6-7743-4d37-934c-f79e30827e4e None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:48.877044 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5cd993c6-7743-4d37-934c-f79e30827e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:48.883238 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61dfd9-edae-447a-916c-5233e2cc3ed0 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:48.883533 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61dfd9-edae-447a-916c-5233e2cc3ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:48.883814 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61dfd9-edae-447a-916c-5233e2cc3ed0 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:48.885317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 492/991] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:46:48.890778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aa9ef-e6a0-48ba-85eb-b0868ca7efe4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:48.891084 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aa9ef-e6a0-48ba-85eb-b0868ca7efe4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:48.891523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aa9ef-e6a0-48ba-85eb-b0868ca7efe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:48.914282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-462aa9ef-e6a0-48ba-85eb-b0868ca7efe4 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:48.915509 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-462aa9ef-e6a0-48ba-85eb-b0868ca7efe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:48.954692 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 500/992] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:48.978801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd993c6-7743-4d37-934c-f79e30827e4e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:49.002618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 493/993] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:49.030304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462aa9ef-e6a0-48ba-85eb-b0868ca7efe4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:49.066464 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:49.078632 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:49.100701 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd993c6-7743-4d37-934c-f79e30827e4e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:49.137993 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4d9a5-ab6e-4e40-964d-b72a7ba378fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:49.138829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4d9a5-ab6e-4e40-964d-b72a7ba378fd None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:49.139114 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4d9a5-ab6e-4e40-964d-b72a7ba378fd None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:49.139389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4d9a5-ab6e-4e40-964d-b72a7ba378fd None admin] PATH_INFO: `/v3/services/compute` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:49.139650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462aa9ef-e6a0-48ba-85eb-b0868ca7efe4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:49.140042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4d9a5-ab6e-4e40-964d-b72a7ba378fd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:49.141406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4d9a5-ab6e-4e40-964d-b72a7ba378fd None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:49.154706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-2da4d9a5-ab6e-4e40-964d-b72a7ba378fd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:46:49.154706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:49.154706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:49.154706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:49.154706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:49.154706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:49.154706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:49.154706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:49.154706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:46:49.154706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:49.154706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:49.154706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:49.154706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:49.154706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:49.154706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:49.154706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:49.154706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:49.155754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:49.155754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:49.155754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:49.155754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:49.155754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:49.155754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:49.155754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:49.155754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:49.155754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:49.155754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:49.155754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:49.155754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:49.155754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:49.155754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:49.155754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:49.155754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:49.156594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:49.156594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:49.156594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:49.156594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:49.156594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:49.156594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:49.156594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:49.156594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:49.156594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:49.156594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:49.156594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:49.156594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:49.156594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:46:49.156594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:49.160958 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 501/994] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:49.161784 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4d9a5-ab6e-4e40-964d-b72a7ba378fd None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:49.180885 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fc75c-442d-47a5-8f5b-2a0277998c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:49.181740 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fc75c-442d-47a5-8f5b-2a0277998c45 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:49.182029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fc75c-442d-47a5-8f5b-2a0277998c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:49.182302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fc75c-442d-47a5-8f5b-2a0277998c45 None admin] PATH_INFO: `/v3/services/image` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:49.182989 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fc75c-442d-47a5-8f5b-2a0277998c45 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:49.184545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fc75c-442d-47a5-8f5b-2a0277998c45 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:49.197754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-6a4fc75c-442d-47a5-8f5b-2a0277998c45 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:46:49.197754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:49.197754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:49.197754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:49.197754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:49.197754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:49.197754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:49.197754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:49.197754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:46:49.197754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:49.197754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:49.197754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:49.197754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:49.197754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:49.197754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:49.197754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:49.197754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:49.198798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:49.198798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:49.198798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:49.198798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:49.198798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:49.198798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:49.198798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:49.198798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:49.198798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:49.198798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:49.198798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:49.198798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:49.198798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:49.198798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:49.198798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:49.198798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:49.199768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:49.199768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:49.199768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:49.199768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:49.199768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:49.199768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:49.199768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:49.199768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:49.199768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:49.199768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:49.199768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:49.199768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:49.199768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:46:49.199768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:49.203948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 494/995] 10.222.0.70 () {56 vars in 1105 bytes} [Thu Jun 16 18:46:49 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:49.204780 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fc75c-442d-47a5-8f5b-2a0277998c45 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:49.264888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4d9a5-ab6e-4e40-964d-b72a7ba378fd None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:49.301238 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844521d1-b474-42f1-b519-a0728393f360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:49.301960 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844521d1-b474-42f1-b519-a0728393f360 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:49.302247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844521d1-b474-42f1-b519-a0728393f360 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:49.302528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844521d1-b474-42f1-b519-a0728393f360 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:49.303251 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844521d1-b474-42f1-b519-a0728393f360 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:49.303563 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844521d1-b474-42f1-b519-a0728393f360 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:49.304930 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844521d1-b474-42f1-b519-a0728393f360 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:49.306066 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:49.310253 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 502/996] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Jun 16 18:46:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:49.310429 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fc75c-442d-47a5-8f5b-2a0277998c45 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:49.315958 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844521d1-b474-42f1-b519-a0728393f360 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:49.328951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:49.349723 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e88511-6633-4b6f-8667-263ba2e88f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:49.350353 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e88511-6633-4b6f-8667-263ba2e88f75 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:49.350756 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e88511-6633-4b6f-8667-263ba2e88f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:49.350979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e88511-6633-4b6f-8667-263ba2e88f75 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:49.351679 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e88511-6633-4b6f-8667-263ba2e88f75 None admin] RBAC: Adding query filter params (name=image) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:49.351985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e88511-6633-4b6f-8667-263ba2e88f75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:49.353329 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e88511-6633-4b6f-8667-263ba2e88f75 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:49.359296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 495/997] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:49.364924 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e88511-6633-4b6f-8667-263ba2e88f75 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:49.436595 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844521d1-b474-42f1-b519-a0728393f360 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:49.473070 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1de4e1-2263-4891-b8e1-3653252abd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:49.474404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1de4e1-2263-4891-b8e1-3653252abd3c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:49.474725 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1de4e1-2263-4891-b8e1-3653252abd3c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:49.475261 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1de4e1-2263-4891-b8e1-3653252abd3c None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:49.475705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1de4e1-2263-4891-b8e1-3653252abd3c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:49.476013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1de4e1-2263-4891-b8e1-3653252abd3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:49.477378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1de4e1-2263-4891-b8e1-3653252abd3c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:49.487032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 503/998] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Jun 16 18:46:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:49.487182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e88511-6633-4b6f-8667-263ba2e88f75 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:49.488895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1de4e1-2263-4891-b8e1-3653252abd3c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:49.524896 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48b51e-6950-4612-902f-8ba90a73eb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:49.525750 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48b51e-6950-4612-902f-8ba90a73eb70 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:49.526042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48b51e-6950-4612-902f-8ba90a73eb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:49.526320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48b51e-6950-4612-902f-8ba90a73eb70 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:49.527060 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48b51e-6950-4612-902f-8ba90a73eb70 None admin] RBAC: Adding query filter params (type=image) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:49.527368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48b51e-6950-4612-902f-8ba90a73eb70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:49.528733 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48b51e-6950-4612-902f-8ba90a73eb70 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:49.538657 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 496/999] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:49 2022] GET /identity/v3/services?type=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:49.580451 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:49.617864 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1de4e1-2263-4891-b8e1-3653252abd3c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:49.654709 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6134881-9ef6-4982-9c62-f19002c555f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:49.655392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6134881-9ef6-4982-9c62-f19002c555f1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:49.655678 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6134881-9ef6-4982-9c62-f19002c555f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:49.655958 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6134881-9ef6-4982-9c62-f19002c555f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:49.656759 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6134881-9ef6-4982-9c62-f19002c555f1 None admin] RBAC: Adding query filter params (interface=public, service_id=7592c13fdaf64faea472c7a487465c28, region_id=RegionOne) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:49.657075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6134881-9ef6-4982-9c62-f19002c555f1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:49.658478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6134881-9ef6-4982-9c62-f19002c555f1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:49.669678 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 504/1000] 10.222.0.70 () {56 vars in 1251 bytes} [Thu Jun 16 18:46:49 2022] GET /identity/v3/endpoints?service_id=7592c13fdaf64faea472c7a487465c28&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:49.671300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48b51e-6950-4612-902f-8ba90a73eb70 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:49.710981 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:49.788638 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48b51e-6950-4612-902f-8ba90a73eb70 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:49.826716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb48dd-3f0c-43e6-a95a-646502c8f38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:49.827399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb48dd-3f0c-43e6-a95a-646502c8f38c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:49.827686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb48dd-3f0c-43e6-a95a-646502c8f38c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:49.827966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb48dd-3f0c-43e6-a95a-646502c8f38c None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:49.828649 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb48dd-3f0c-43e6-a95a-646502c8f38c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:49.830008 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb48dd-3f0c-43e6-a95a-646502c8f38c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:49.842614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 497/1001] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Jun 16 18:46:49 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:46:50.053437 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbe94b-51db-41d8-b4b6-8a6df85c0b8b None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:50.054007 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbe94b-51db-41d8-b4b6-8a6df85c0b8b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:50.054007 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbe94b-51db-41d8-b4b6-8a6df85c0b8b None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:50.055519 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 505/1002] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:50.060670 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6a1d4-c5d2-466c-ac4f-a77fea6fe6c5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:50.061057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6a1d4-c5d2-466c-ac4f-a77fea6fe6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:50.061229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6a1d4-c5d2-466c-ac4f-a77fea6fe6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:50.078305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5ec6a1d4-c5d2-466c-ac4f-a77fea6fe6c5 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:50.079499 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5ec6a1d4-c5d2-466c-ac4f-a77fea6fe6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:50.148618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 498/1003] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:50.171585 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbe94b-51db-41d8-b4b6-8a6df85c0b8b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:50.273821 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbe94b-51db-41d8-b4b6-8a6df85c0b8b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:50.307466 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec8b08-8983-40b7-9e9a-9a5e7c671540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:50.309848 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec8b08-8983-40b7-9e9a-9a5e7c671540 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:50.309848 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec8b08-8983-40b7-9e9a-9a5e7c671540 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:50.309848 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec8b08-8983-40b7-9e9a-9a5e7c671540 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:50.309848 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec8b08-8983-40b7-9e9a-9a5e7c671540 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:50.310813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec8b08-8983-40b7-9e9a-9a5e7c671540 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:50.313584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:50.325229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-6eec8b08-8983-40b7-9e9a-9a5e7c671540 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:46:50.325229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:50.325229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:50.325229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:50.325229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:50.325229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:50.325229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:50.325229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:50.325229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:46:50.325229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:50.325229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:50.325229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:50.325229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:50.325229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:50.325229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:50.325229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:50.325229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:50.326303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:50.326303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:50.326303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:50.326303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:50.326303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:50.326303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:50.326303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:50.326303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:50.326303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:50.326303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:50.326303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:50.326303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:50.326303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:50.326303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:50.326303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:50.326303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:50.327187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:50.327187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:50.327187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:50.327187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:50.327187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:50.327187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:50.327187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:50.327187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:50.327187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:50.327187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:50.327187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:50.327187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:50.327187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:46:50.327187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:50.331291 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 506/1004] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Jun 16 18:46:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:50.332556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6a1d4-c5d2-466c-ac4f-a77fea6fe6c5 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:50.354496 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:50.434965 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6a1d4-c5d2-466c-ac4f-a77fea6fe6c5 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:50.468345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce1220-86c2-4f75-b368-768ec4e97d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:50.469004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce1220-86c2-4f75-b368-768ec4e97d2b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:50.469284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce1220-86c2-4f75-b368-768ec4e97d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:50.469561 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce1220-86c2-4f75-b368-768ec4e97d2b None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:50.470257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce1220-86c2-4f75-b368-768ec4e97d2b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:50.470558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce1220-86c2-4f75-b368-768ec4e97d2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:50.471929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce1220-86c2-4f75-b368-768ec4e97d2b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:50.482711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 499/1005] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Jun 16 18:46:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:50.484311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec8b08-8983-40b7-9e9a-9a5e7c671540 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:50.586366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec8b08-8983-40b7-9e9a-9a5e7c671540 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:50.620088 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df934f8b-45cb-4682-9290-442ec4cf0b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:50.620753 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df934f8b-45cb-4682-9290-442ec4cf0b70 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:50.621035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df934f8b-45cb-4682-9290-442ec4cf0b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:50.621306 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df934f8b-45cb-4682-9290-442ec4cf0b70 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:50.622012 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df934f8b-45cb-4682-9290-442ec4cf0b70 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:50.622318 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df934f8b-45cb-4682-9290-442ec4cf0b70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:50.623701 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df934f8b-45cb-4682-9290-442ec4cf0b70 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:50.634060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 507/1006] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Jun 16 18:46:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:50.634957 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce1220-86c2-4f75-b368-768ec4e97d2b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:50.723272 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:50.741238 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce1220-86c2-4f75-b368-768ec4e97d2b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:50.777618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9b64d-fc84-4256-a201-314b992c2a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:50.778283 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9b64d-fc84-4256-a201-314b992c2a3b None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:50.778590 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9b64d-fc84-4256-a201-314b992c2a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:50.778870 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9b64d-fc84-4256-a201-314b992c2a3b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:50.779625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9b64d-fc84-4256-a201-314b992c2a3b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:50.780989 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9b64d-fc84-4256-a201-314b992c2a3b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:50.802612 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 500/1007] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Jun 16 18:46:50 2022] POST /identity/v3/endpoints => generated 325 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:46:51.726846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d18b1-67e3-4e97-a73d-36395c69ede6 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:51.727408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d18b1-67e3-4e97-a73d-36395c69ede6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:51.727408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d18b1-67e3-4e97-a73d-36395c69ede6 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:51.728769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 508/1008] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:51.734337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36305f8f-a6f0-4261-9b48-939999646779 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:51.734793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36305f8f-a6f0-4261-9b48-939999646779 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:51.734963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36305f8f-a6f0-4261-9b48-939999646779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:51.752055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-36305f8f-a6f0-4261-9b48-939999646779 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:51.753230 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-36305f8f-a6f0-4261-9b48-939999646779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:51.826968 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 501/1009] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:51 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:51.850466 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d18b1-67e3-4e97-a73d-36395c69ede6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:51.916656 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:51.965260 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d18b1-67e3-4e97-a73d-36395c69ede6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:52.004847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997607c-231e-4e33-a773-304e267a9a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:52.005590 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997607c-231e-4e33-a773-304e267a9a26 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:52.005874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997607c-231e-4e33-a773-304e267a9a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:52.006153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997607c-231e-4e33-a773-304e267a9a26 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:52.006909 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997607c-231e-4e33-a773-304e267a9a26 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:52.008269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997607c-231e-4e33-a773-304e267a9a26 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:52.024656 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 509/1010] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Jun 16 18:46:51 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:46:52.040648 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e04d0e-f6ae-4690-b3a6-9d4b76fc3bce None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:52.041037 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e04d0e-f6ae-4690-b3a6-9d4b76fc3bce None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:52.041219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e04d0e-f6ae-4690-b3a6-9d4b76fc3bce None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:52.043003 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 502/1011] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:52.048654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6a058-2cf2-43e0-a426-6ec4c49c2605 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:52.048969 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6a058-2cf2-43e0-a426-6ec4c49c2605 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:52.049236 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6a058-2cf2-43e0-a426-6ec4c49c2605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:52.068808 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-41f6a058-2cf2-43e0-a426-6ec4c49c2605 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:52.070024 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-41f6a058-2cf2-43e0-a426-6ec4c49c2605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:52.154611 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 510/1012] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:52 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:52.174445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e04d0e-f6ae-4690-b3a6-9d4b76fc3bce None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:52.250676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:52.286621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e04d0e-f6ae-4690-b3a6-9d4b76fc3bce None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:52.321707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551ca1a-3f9d-4b04-ae25-5b43e1d8b614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:52.322529 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551ca1a-3f9d-4b04-ae25-5b43e1d8b614 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:52.322855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551ca1a-3f9d-4b04-ae25-5b43e1d8b614 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:52.323139 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551ca1a-3f9d-4b04-ae25-5b43e1d8b614 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:52.323842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551ca1a-3f9d-4b04-ae25-5b43e1d8b614 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:52.325233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551ca1a-3f9d-4b04-ae25-5b43e1d8b614 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:52.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-d551ca1a-3f9d-4b04-ae25-5b43e1d8b614 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:46:52.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:52.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:52.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:52.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:52.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:52.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:52.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:52.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:46:52.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:52.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:52.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:52.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:52.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:52.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:52.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:52.337873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:52.339159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:52.339159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:52.339159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:52.339159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:52.339159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:52.339159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:52.339159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:52.339159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:52.339159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:52.339159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:52.339159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:52.339159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:52.339159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:52.339159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:52.339159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:52.339159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:52.340007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:52.340007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:52.340007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:52.340007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:52.340007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:52.340007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:52.340007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:52.340007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:52.340007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:52.340007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:52.340007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:52.340007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:52.340007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:46:52.340007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:52.340007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 503/1013] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:52.346759 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6a058-2cf2-43e0-a426-6ec4c49c2605 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:52.400324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:52.452982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6a058-2cf2-43e0-a426-6ec4c49c2605 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:52.490176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32219e3-1144-4d43-a161-afd794af1504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:52.491062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32219e3-1144-4d43-a161-afd794af1504 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:52.491474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32219e3-1144-4d43-a161-afd794af1504 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:52.491881 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32219e3-1144-4d43-a161-afd794af1504 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:52.492727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32219e3-1144-4d43-a161-afd794af1504 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:52.493187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32219e3-1144-4d43-a161-afd794af1504 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:52.494736 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32219e3-1144-4d43-a161-afd794af1504 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:52.504276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 511/1014] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Jun 16 18:46:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:52.505563 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551ca1a-3f9d-4b04-ae25-5b43e1d8b614 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:52.610752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551ca1a-3f9d-4b04-ae25-5b43e1d8b614 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:52.644521 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500cf0c-b53b-4ece-8f14-0c40b481895c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:52.645728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500cf0c-b53b-4ece-8f14-0c40b481895c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:52.646010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500cf0c-b53b-4ece-8f14-0c40b481895c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:52.646284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500cf0c-b53b-4ece-8f14-0c40b481895c None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:52.647017 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500cf0c-b53b-4ece-8f14-0c40b481895c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:52.647323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500cf0c-b53b-4ece-8f14-0c40b481895c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:52.648676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500cf0c-b53b-4ece-8f14-0c40b481895c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:52.649900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:52.660178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32219e3-1144-4d43-a161-afd794af1504 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:52.661452 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 504/1015] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Jun 16 18:46:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:52.762891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32219e3-1144-4d43-a161-afd794af1504 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:52.778002 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:52.799674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15746c1f-b89f-4258-979f-9f21f59fe308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:52.800337 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15746c1f-b89f-4258-979f-9f21f59fe308 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:52.800618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15746c1f-b89f-4258-979f-9f21f59fe308 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:52.800894 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15746c1f-b89f-4258-979f-9f21f59fe308 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:52.801771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15746c1f-b89f-4258-979f-9f21f59fe308 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:52.803169 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15746c1f-b89f-4258-979f-9f21f59fe308 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:52.829567 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 512/1016] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Jun 16 18:46:52 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:46:52.984708 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03cdac-042d-4c92-a82d-43186736d522 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:52.985229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03cdac-042d-4c92-a82d-43186736d522 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:52.985346 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03cdac-042d-4c92-a82d-43186736d522 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:52.986860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 505/1017] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:52.992334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aeefa-0e95-4f29-b463-1a7f07dc35b9 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:52.992632 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aeefa-0e95-4f29-b463-1a7f07dc35b9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:52.992915 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aeefa-0e95-4f29-b463-1a7f07dc35b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:53.011120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-462aeefa-0e95-4f29-b463-1a7f07dc35b9 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:53.012305 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-462aeefa-0e95-4f29-b463-1a7f07dc35b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:53.094656 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 513/1018] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:53.119120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03cdac-042d-4c92-a82d-43186736d522 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:53.228119 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03cdac-042d-4c92-a82d-43186736d522 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:53.262438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec503c-6c23-4b09-9605-e0e6d03c247b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:53.263265 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec503c-6c23-4b09-9605-e0e6d03c247b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:53.263552 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec503c-6c23-4b09-9605-e0e6d03c247b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:53.263823 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec503c-6c23-4b09-9605-e0e6d03c247b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:53.265027 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec503c-6c23-4b09-9605-e0e6d03c247b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:53.266442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec503c-6c23-4b09-9605-e0e6d03c247b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:53.268547 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 506/1019] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:53.275051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462aeefa-0e95-4f29-b463-1a7f07dc35b9 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:53.368522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:53.376658 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462aeefa-0e95-4f29-b463-1a7f07dc35b9 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:53.412474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e9571-4879-4e86-93fd-bbe2fd147b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:53.413129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e9571-4879-4e86-93fd-bbe2fd147b80 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:53.413409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e9571-4879-4e86-93fd-bbe2fd147b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:53.413681 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e9571-4879-4e86-93fd-bbe2fd147b80 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:53.414373 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e9571-4879-4e86-93fd-bbe2fd147b80 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:53.415828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e9571-4879-4e86-93fd-bbe2fd147b80 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:53.450975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 514/1020] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Jun 16 18:46:53 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:46:54.226887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e4ebf-126e-4b73-8403-88f383264894 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:54.227381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e4ebf-126e-4b73-8403-88f383264894 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:54.227381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e4ebf-126e-4b73-8403-88f383264894 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:54.228913 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 507/1021] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:54.233869 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb028b-53c6-4cac-b3fe-16c7b89f4ede None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:54.234247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb028b-53c6-4cac-b3fe-16c7b89f4ede None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:54.234435 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb028b-53c6-4cac-b3fe-16c7b89f4ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:54.252184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-a7eb028b-53c6-4cac-b3fe-16c7b89f4ede None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:54.253368 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a7eb028b-53c6-4cac-b3fe-16c7b89f4ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:54.323660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 515/1022] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:54.345922 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e4ebf-126e-4b73-8403-88f383264894 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:54.416479 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:54.453448 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e4ebf-126e-4b73-8403-88f383264894 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:54.486803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c6106-8940-4eeb-9b7a-ddb363b34b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:54.487525 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c6106-8940-4eeb-9b7a-ddb363b34b3f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:54.487808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c6106-8940-4eeb-9b7a-ddb363b34b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:54.488083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c6106-8940-4eeb-9b7a-ddb363b34b3f None admin] PATH_INFO: `/v3/services/image` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:54.488780 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c6106-8940-4eeb-9b7a-ddb363b34b3f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:54.490134 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c6106-8940-4eeb-9b7a-ddb363b34b3f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:54.501868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-560c6106-8940-4eeb-9b7a-ddb363b34b3f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:46:54.501868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:54.501868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:54.501868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:54.501868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:54.501868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:54.501868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:54.501868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:54.501868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:46:54.501868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:54.501868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:54.501868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:54.501868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:54.501868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:54.501868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:54.501868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:54.501868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:54.502880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:54.502880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:54.502880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:54.502880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:54.502880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:54.502880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:54.502880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:54.502880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:54.502880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:54.502880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:54.502880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:54.502880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:54.502880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:54.502880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:54.502880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:54.502880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:54.503742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:54.503742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:54.503742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:54.503742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:54.503742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:54.503742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:54.503742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:54.503742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:54.503742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:54.503742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:54.503742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:54.503742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:54.503742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:46:54.503742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:54.506902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 508/1023] 10.222.0.70 () {56 vars in 1105 bytes} [Thu Jun 16 18:46:54 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:54.509753 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb028b-53c6-4cac-b3fe-16c7b89f4ede None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:54.591929 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:54.617228 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb028b-53c6-4cac-b3fe-16c7b89f4ede None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:54.652373 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ed806-e471-4bac-87af-d2015535d47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:54.653048 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ed806-e471-4bac-87af-d2015535d47a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:54.653332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ed806-e471-4bac-87af-d2015535d47a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:54.653608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ed806-e471-4bac-87af-d2015535d47a None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:54.654312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ed806-e471-4bac-87af-d2015535d47a None admin] RBAC: Adding query filter params (name=image) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:54.654645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ed806-e471-4bac-87af-d2015535d47a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:54.656013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ed806-e471-4bac-87af-d2015535d47a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:54.666346 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 516/1024] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:54.667278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c6106-8940-4eeb-9b7a-ddb363b34b3f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:54.771417 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c6106-8940-4eeb-9b7a-ddb363b34b3f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:54.799723 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:54.807270 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a397a-e3af-4795-9ceb-d8c8481fa32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:54.807940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a397a-e3af-4795-9ceb-d8c8481fa32c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:54.808221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a397a-e3af-4795-9ceb-d8c8481fa32c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:54.808494 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a397a-e3af-4795-9ceb-d8c8481fa32c None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:54.809197 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a397a-e3af-4795-9ceb-d8c8481fa32c None admin] RBAC: Adding query filter params (type=image) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:54.809504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a397a-e3af-4795-9ceb-d8c8481fa32c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:54.810894 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a397a-e3af-4795-9ceb-d8c8481fa32c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:54.821416 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 509/1025] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:54.822681 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ed806-e471-4bac-87af-d2015535d47a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:54.928097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ed806-e471-4bac-87af-d2015535d47a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:54.974154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade21651-12f4-4fb6-a44e-8ad9df51f33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:54.974845 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade21651-12f4-4fb6-a44e-8ad9df51f33b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:54.975133 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade21651-12f4-4fb6-a44e-8ad9df51f33b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:54.975410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade21651-12f4-4fb6-a44e-8ad9df51f33b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:54.976199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade21651-12f4-4fb6-a44e-8ad9df51f33b None admin] RBAC: Adding query filter params (interface=public, service_id=feefe790377846afbfd7cafbeaf480af, region_id=RegionOne) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:54.976513 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade21651-12f4-4fb6-a44e-8ad9df51f33b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:54.977876 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade21651-12f4-4fb6-a44e-8ad9df51f33b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:54.988256 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 517/1026] 10.222.0.70 () {56 vars in 1251 bytes} [Thu Jun 16 18:46:54 2022] GET /identity/v3/endpoints?service_id=feefe790377846afbfd7cafbeaf480af&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:54.989331 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a397a-e3af-4795-9ceb-d8c8481fa32c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:55.097057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a397a-e3af-4795-9ceb-d8c8481fa32c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:55.135165 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0f3cb-54b7-4efd-b29f-6361f88b8d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:55.135865 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0f3cb-54b7-4efd-b29f-6361f88b8d48 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:55.136149 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0f3cb-54b7-4efd-b29f-6361f88b8d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:55.136462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0f3cb-54b7-4efd-b29f-6361f88b8d48 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:55.137174 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0f3cb-54b7-4efd-b29f-6361f88b8d48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:55.138600 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0f3cb-54b7-4efd-b29f-6361f88b8d48 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:55.146426 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 510/1027] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Jun 16 18:46:54 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:46:55.172115 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8886c02-e751-4651-9f93-ac4b0b5e2cf0 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:55.172400 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8886c02-e751-4651-9f93-ac4b0b5e2cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:55.172846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8886c02-e751-4651-9f93-ac4b0b5e2cf0 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:55.174272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 518/1028] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:55.181329 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfd837-b351-4157-921b-006e269f2454 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:55.181642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfd837-b351-4157-921b-006e269f2454 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:55.181901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfd837-b351-4157-921b-006e269f2454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:55.200066 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-99bfd837-b351-4157-921b-006e269f2454 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:55.201290 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-99bfd837-b351-4157-921b-006e269f2454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:55.260983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:55.278328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 511/1029] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:55.303072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8886c02-e751-4651-9f93-ac4b0b5e2cf0 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:55.312755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:55.413079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8886c02-e751-4651-9f93-ac4b0b5e2cf0 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:55.448150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965ad54-de01-4efe-8a60-ab2442e19538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:55.448962 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965ad54-de01-4efe-8a60-ab2442e19538 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:55.449248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965ad54-de01-4efe-8a60-ab2442e19538 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:55.449527 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965ad54-de01-4efe-8a60-ab2442e19538 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:55.460566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965ad54-de01-4efe-8a60-ab2442e19538 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:55.468744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-c965ad54-de01-4efe-8a60-ab2442e19538 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:46:55.468744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:55.468744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:55.468744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:55.468744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:55.468744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:55.468744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:55.468744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:55.468744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:55.468744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:55.468744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:55.468744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:55.468744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:55.468744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:55.468744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:55.468744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:55.469700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:55.469700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:55.469700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:55.469700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:55.469700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:55.469700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:55.469700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:55.469700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:55.469700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:55.469700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:55.469700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:55.469700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:55.469700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:55.469700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:55.469700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:55.469700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:55.470562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:55.470562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:55.470562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:55.470562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:55.470562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:55.470562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:55.470562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:55.470562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:55.470562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:55.470562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:55.470562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:46:55.470562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:55.470562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 519/1030] 10.222.0.70 () {56 vars in 1120 bytes} [Thu Jun 16 18:46:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:55.476209 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfd837-b351-4157-921b-006e269f2454 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:55.578185 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfd837-b351-4157-921b-006e269f2454 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:55.612488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1065be2-260a-43c8-af62-b04ff1ac4eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:55.613219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1065be2-260a-43c8-af62-b04ff1ac4eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:55.614133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1065be2-260a-43c8-af62-b04ff1ac4eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:55.614687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1065be2-260a-43c8-af62-b04ff1ac4eb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:55.615494 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1065be2-260a-43c8-af62-b04ff1ac4eb5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:55.615812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1065be2-260a-43c8-af62-b04ff1ac4eb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:55.617181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1065be2-260a-43c8-af62-b04ff1ac4eb5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:55.632214 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 512/1031] 10.222.0.70 () {56 vars in 1115 bytes} [Thu Jun 16 18:46:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:55.633532 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965ad54-de01-4efe-8a60-ab2442e19538 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:55.700292 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e8bdc-9f12-43d3-9b22-743438cfccce None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:55.700612 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e8bdc-9f12-43d3-9b22-743438cfccce None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:55.700863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e8bdc-9f12-43d3-9b22-743438cfccce None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:55.702361 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 513/1032] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:46:55.704496 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:55.707395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecb63c-6802-45f3-a2ae-b8779876f7e4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:55.708358 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecb63c-6802-45f3-a2ae-b8779876f7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:55.708643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecb63c-6802-45f3-a2ae-b8779876f7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:55.731138 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f2ecb63c-6802-45f3-a2ae-b8779876f7e4 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:55.732333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f2ecb63c-6802-45f3-a2ae-b8779876f7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:55.743366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965ad54-de01-4efe-8a60-ab2442e19538 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:55.779736 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c38f5-6d95-45cc-9129-024573fe53ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:55.780832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c38f5-6d95-45cc-9129-024573fe53ef None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:55.780832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c38f5-6d95-45cc-9129-024573fe53ef None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:55.781038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:55.781118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c38f5-6d95-45cc-9129-024573fe53ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:55.793503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c38f5-6d95-45cc-9129-024573fe53ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:55.794984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c38f5-6d95-45cc-9129-024573fe53ef None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:55.804589 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 514/1033] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:55.813723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-497c38f5-6d95-45cc-9129-024573fe53ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:55.813723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:55.813723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:55.813723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:55.813723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:55.813723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:55.813723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:55.813723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:55.813723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:55.813723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:55.813723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:55.813723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:55.813723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:55.813723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:55.813723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:55.814643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:55.814643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:55.814643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:55.814643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:55.814643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:55.814643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:55.814643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:55.814643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:55.814643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:55.814643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:55.814643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:55.814643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:55.814643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:55.814643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:55.814643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:55.814643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:55.814643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:55.815556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:55.815556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:55.815556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:55.815556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:46:55.818888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 520/1034] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:55.821830 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ecb63c-6802-45f3-a2ae-b8779876f7e4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:55.831147 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c38f5-6d95-45cc-9129-024573fe53ef None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:55.939063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ecb63c-6802-45f3-a2ae-b8779876f7e4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:55.951012 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c38f5-6d95-45cc-9129-024573fe53ef None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:55.973368 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:55.977120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58f060-d938-4c08-89c0-b27382bcaa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:55.977731 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58f060-d938-4c08-89c0-b27382bcaa41 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:55.978013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58f060-d938-4c08-89c0-b27382bcaa41 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:55.978290 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58f060-d938-4c08-89c0-b27382bcaa41 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:55.979075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58f060-d938-4c08-89c0-b27382bcaa41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:55.979414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58f060-d938-4c08-89c0-b27382bcaa41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:55.980747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58f060-d938-4c08-89c0-b27382bcaa41 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:55.993069 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62233963-f0c8-4979-80ed-1bd9a4fe440d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:55.993767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 515/1035] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:55.998775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62233963-f0c8-4979-80ed-1bd9a4fe440d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:55.999222 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62233963-f0c8-4979-80ed-1bd9a4fe440d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:55.999393 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62233963-f0c8-4979-80ed-1bd9a4fe440d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:56.000028 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58f060-d938-4c08-89c0-b27382bcaa41 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:56.000554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62233963-f0c8-4979-80ed-1bd9a4fe440d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:56.001982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62233963-f0c8-4979-80ed-1bd9a4fe440d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:56.004168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 521/1036] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:56.009075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62233963-f0c8-4979-80ed-1bd9a4fe440d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:56.053704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:56.116863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58f060-d938-4c08-89c0-b27382bcaa41 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:56.123324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62233963-f0c8-4979-80ed-1bd9a4fe440d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:56.151960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f2db0b-77dc-429b-957c-cf470a2cc654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:56.152751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2db0b-77dc-429b-957c-cf470a2cc654 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:56.153034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2db0b-77dc-429b-957c-cf470a2cc654 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:56.153386 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2db0b-77dc-429b-957c-cf470a2cc654 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:56.159536 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b1b4a-f29b-49d7-a8af-d4171c2d7ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:56.160209 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b1b4a-f29b-49d7-a8af-d4171c2d7ea6 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:56.161103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b1b4a-f29b-49d7-a8af-d4171c2d7ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:56.161360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b1b4a-f29b-49d7-a8af-d4171c2d7ea6 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:56.162021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2db0b-77dc-429b-957c-cf470a2cc654 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:56.162128 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b1b4a-f29b-49d7-a8af-d4171c2d7ea6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:56.163480 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b1b4a-f29b-49d7-a8af-d4171c2d7ea6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:56.163579 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2db0b-77dc-429b-957c-cf470a2cc654 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:56.179645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-48f2db0b-77dc-429b-957c-cf470a2cc654 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:46:56.179645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:56.179645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:56.179645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:56.179645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:56.179645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:56.179645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:56.179645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:56.179645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:56.179645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:56.179645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:56.179645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:56.179645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:46:56.179645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:46:56.179645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:46:56.179645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:56.180725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:56.181716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:56.181716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:56.181716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:46:56.181716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:46:56.181716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:46:56.181716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:46:56.181716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:46:56.181716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:46:56.181716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:46:56.181716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:56.181716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 516/1037] 10.222.0.70 () {56 vars in 1093 bytes} [Thu Jun 16 18:46:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:56.186486 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f2db0b-77dc-429b-957c-cf470a2cc654 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:56.189446 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 522/1038] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:46:56 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:46:56.304214 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:56.308434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f2db0b-77dc-429b-957c-cf470a2cc654 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:56.347049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3266891-74da-4bb1-954e-5e7205be8c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:56.347768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3266891-74da-4bb1-954e-5e7205be8c99 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:56.348738 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3266891-74da-4bb1-954e-5e7205be8c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:56.349018 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3266891-74da-4bb1-954e-5e7205be8c99 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:56.349880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3266891-74da-4bb1-954e-5e7205be8c99 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:56.350199 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3266891-74da-4bb1-954e-5e7205be8c99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:56.351714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3266891-74da-4bb1-954e-5e7205be8c99 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:56.372147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 517/1039] 10.222.0.70 () {56 vars in 1133 bytes} [Thu Jun 16 18:46:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:56.377802 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b1b4a-f29b-49d7-a8af-d4171c2d7ea6 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:56.428387 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:56.492555 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b1b4a-f29b-49d7-a8af-d4171c2d7ea6 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:56.527558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b762561-e9d1-4079-805b-043727ee7dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:56.528401 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b762561-e9d1-4079-805b-043727ee7dbb None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:56.528782 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b762561-e9d1-4079-805b-043727ee7dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:56.529143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b762561-e9d1-4079-805b-043727ee7dbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:56.602611 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b762561-e9d1-4079-805b-043727ee7dbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:56.604076 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b762561-e9d1-4079-805b-043727ee7dbb None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:56.646367 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:56.708294 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 523/1040] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:46:56 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:46:56.714123 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3266891-74da-4bb1-954e-5e7205be8c99 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:56.820358 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3266891-74da-4bb1-954e-5e7205be8c99 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:56.858617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ad04c-0369-4f57-aece-cbb23c920ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:56.859335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ad04c-0369-4f57-aece-cbb23c920ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:56.859616 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ad04c-0369-4f57-aece-cbb23c920ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:56.859888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ad04c-0369-4f57-aece-cbb23c920ad6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:56.870095 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ad04c-0369-4f57-aece-cbb23c920ad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:56.871750 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ad04c-0369-4f57-aece-cbb23c920ad6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:56.890693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-736ad04c-0369-4f57-aece-cbb23c920ad6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:56.890693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:56.890693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:56.890693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:56.890693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:56.890693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:56.890693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:56.890693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:56.890693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:56.890693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:56.890693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:56.890693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:56.890693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:46:56.890693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:46:56.890693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:46:56.892078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:46:56.892078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:56.892078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:56.892078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:56.892078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:56.892078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:56.892078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:56.892078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:56.892078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:56.892078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:56.892078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:56.892078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:56.892078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:56.892078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:56.892078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:56.892078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:56.892078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:56.893183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:46:56.893183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:46:56.893183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:46:56.893183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:56.893183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 518/1041] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:56.898932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b762561-e9d1-4079-805b-043727ee7dbb None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:57.013674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b762561-e9d1-4079-805b-043727ee7dbb None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:57.048699 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69dbf9-a0d6-4a90-a694-3b8803fd0ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:57.049602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69dbf9-a0d6-4a90-a694-3b8803fd0ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:57.050028 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69dbf9-a0d6-4a90-a694-3b8803fd0ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:57.050445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69dbf9-a0d6-4a90-a694-3b8803fd0ed3 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:57.051989 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69dbf9-a0d6-4a90-a694-3b8803fd0ed3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:57.052440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69dbf9-a0d6-4a90-a694-3b8803fd0ed3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:57.053941 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69dbf9-a0d6-4a90-a694-3b8803fd0ed3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:57.066201 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 524/1042] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:57.072142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ad04c-0369-4f57-aece-cbb23c920ad6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:57.085563 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:57.182153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ad04c-0369-4f57-aece-cbb23c920ad6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:57.217360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b0334f-fdac-4147-94dc-50bfabd8203c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:57.218151 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0334f-fdac-4147-94dc-50bfabd8203c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:57.218436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0334f-fdac-4147-94dc-50bfabd8203c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:57.218743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0334f-fdac-4147-94dc-50bfabd8203c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:57.225619 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b0334f-fdac-4147-94dc-50bfabd8203c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:57.227100 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b0334f-fdac-4147-94dc-50bfabd8203c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:57.241367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b0b0334f-fdac-4147-94dc-50bfabd8203c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:57.241367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:57.241367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:57.241367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:57.241367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:57.241367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:57.241367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:57.241367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:57.241367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:57.241367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:57.241367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:57.241367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:57.241367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:46:57.241367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:46:57.241367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:46:57.242280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:46:57.242280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:57.242280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:57.242280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:57.242280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:57.242280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:57.242280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:57.242280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:57.242280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:57.242280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:57.242280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:57.242280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:57.242280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:57.242280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:57.242280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:57.242280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:57.242280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:57.243767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:46:57.243767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:46:57.243767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:46:57.243767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:46:57.243767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:46:57.243767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:46:57.243767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:46:57.243767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:57.244498 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 519/1043] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:46:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:57.248453 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69dbf9-a0d6-4a90-a694-3b8803fd0ed3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:57.251589 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:57.343468 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2bf5a-50b2-45d3-8c4c-e3b01a1029da None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:57.343991 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2bf5a-50b2-45d3-8c4c-e3b01a1029da None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:57.343991 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2bf5a-50b2-45d3-8c4c-e3b01a1029da None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:57.345587 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 520/1044] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:57.350745 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d3fdc-bff0-4178-a5eb-5252bff153c2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:57.351032 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d3fdc-bff0-4178-a5eb-5252bff153c2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:57.351306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d3fdc-bff0-4178-a5eb-5252bff153c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:57.363007 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69dbf9-a0d6-4a90-a694-3b8803fd0ed3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:57.368308 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-933d3fdc-bff0-4178-a5eb-5252bff153c2 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:57.369481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-933d3fdc-bff0-4178-a5eb-5252bff153c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:57.402330 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a4c15-4b95-411f-a2bf-e60abcc368d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:57.403053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a4c15-4b95-411f-a2bf-e60abcc368d8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:57.403330 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a4c15-4b95-411f-a2bf-e60abcc368d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:57.403618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a4c15-4b95-411f-a2bf-e60abcc368d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:57.404415 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a4c15-4b95-411f-a2bf-e60abcc368d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:57.404718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a4c15-4b95-411f-a2bf-e60abcc368d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:57.406134 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a4c15-4b95-411f-a2bf-e60abcc368d8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:57.419121 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 525/1045] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:46:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:57.424727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a4c15-4b95-411f-a2bf-e60abcc368d8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:57.443488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:57.454597 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 521/1046] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:46:57.472622 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d3fdc-bff0-4178-a5eb-5252bff153c2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:57.492097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:57.542809 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a4c15-4b95-411f-a2bf-e60abcc368d8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:57.579412 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d3fdc-bff0-4178-a5eb-5252bff153c2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:57.584059 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e2261-0022-4898-88c6-9d9b0831046c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:57.584703 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e2261-0022-4898-88c6-9d9b0831046c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:57.585029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e2261-0022-4898-88c6-9d9b0831046c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:57.585306 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e2261-0022-4898-88c6-9d9b0831046c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:57.586120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e2261-0022-4898-88c6-9d9b0831046c None admin] RBAC: Adding query filter params (role.id=ed8734ab73344d1983b1cdaa7fa3238a, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=8342da838fe4462b9de2817ad3ac4ff7) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:57.586428 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e2261-0022-4898-88c6-9d9b0831046c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:57.587887 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e2261-0022-4898-88c6-9d9b0831046c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:57.593764 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 526/1047] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:46:57 2022] GET /identity/v3/role_assignments?user.id=8342da838fe4462b9de2817ad3ac4ff7&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=ed8734ab73344d1983b1cdaa7fa3238a => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:57.617424 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b014da1-e31f-46c4-ba64-0da7f8b5d87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:57.618249 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b014da1-e31f-46c4-ba64-0da7f8b5d87c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:57.618531 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b014da1-e31f-46c4-ba64-0da7f8b5d87c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:57.619030 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b014da1-e31f-46c4-ba64-0da7f8b5d87c None admin] PATH_INFO: `/v3/services/image` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:57.619706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b014da1-e31f-46c4-ba64-0da7f8b5d87c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:57.621110 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b014da1-e31f-46c4-ba64-0da7f8b5d87c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:57.634343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-8b014da1-e31f-46c4-ba64-0da7f8b5d87c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:46:57.634343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:57.634343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:46:57.634343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:46:57.634343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:46:57.634343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:46:57.634343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:46:57.634343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:57.634343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:46:57.634343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:57.634343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:46:57.634343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:57.634343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:57.634343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:57.634343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:57.634343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:57.634343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:57.635690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:57.635690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:57.635690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:57.635690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:57.635690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:57.635690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:57.635690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:46:57.635690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:46:57.635690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:46:57.635690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:46:57.635690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:57.635690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:57.635690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:57.635690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:57.635690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:57.635690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:57.636574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:57.636574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:57.636574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:57.636574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:57.636574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:57.636574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:57.636574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:57.636574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:57.636574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:57.636574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:57.636574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:46:57.636574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:46:57.636574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:46:57.636574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:57.638968 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 522/1048] 10.222.0.70 () {56 vars in 1105 bytes} [Thu Jun 16 18:46:57 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:46:57.641950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e2261-0022-4898-88c6-9d9b0831046c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:57.757389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e2261-0022-4898-88c6-9d9b0831046c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:57.784405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:57.794187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca310a5-18e6-4496-98ff-3784d5ee3ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:57.794894 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca310a5-18e6-4496-98ff-3784d5ee3ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:57.795180 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca310a5-18e6-4496-98ff-3784d5ee3ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:57.795459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca310a5-18e6-4496-98ff-3784d5ee3ff3 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:57.796165 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca310a5-18e6-4496-98ff-3784d5ee3ff3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:57.796474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca310a5-18e6-4496-98ff-3784d5ee3ff3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:57.797834 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca310a5-18e6-4496-98ff-3784d5ee3ff3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:57.808247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 527/1049] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:57.809682 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b014da1-e31f-46c4-ba64-0da7f8b5d87c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:57.880313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:57.921611 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b014da1-e31f-46c4-ba64-0da7f8b5d87c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:57.964082 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89070263-7339-4a7e-889a-7280d7d47b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:57.964754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89070263-7339-4a7e-889a-7280d7d47b84 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:57.965037 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89070263-7339-4a7e-889a-7280d7d47b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:57.965312 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89070263-7339-4a7e-889a-7280d7d47b84 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:57.966017 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89070263-7339-4a7e-889a-7280d7d47b84 None admin] RBAC: Adding query filter params (type=image) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:57.966323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89070263-7339-4a7e-889a-7280d7d47b84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:57.967725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89070263-7339-4a7e-889a-7280d7d47b84 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:57.973677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 523/1050] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:46:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:57.979522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca310a5-18e6-4496-98ff-3784d5ee3ff3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:58.090233 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca310a5-18e6-4496-98ff-3784d5ee3ff3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:58.124236 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd08be6-3d18-4394-9080-724a426227d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:58.124894 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd08be6-3d18-4394-9080-724a426227d8 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:58.125177 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd08be6-3d18-4394-9080-724a426227d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:58.125453 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd08be6-3d18-4394-9080-724a426227d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:58.126191 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd08be6-3d18-4394-9080-724a426227d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:58.127582 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd08be6-3d18-4394-9080-724a426227d8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:58.154612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 528/1051] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Jun 16 18:46:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:46:58.497221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07ca43-af31-435f-bf23-b30fbdfed630 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:58.497769 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07ca43-af31-435f-bf23-b30fbdfed630 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:58.497769 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07ca43-af31-435f-bf23-b30fbdfed630 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:58.499349 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 524/1052] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:58.505039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2718-2421-48d1-915c-a502044a9949 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:58.505441 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2718-2421-48d1-915c-a502044a9949 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:58.505607 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2718-2421-48d1-915c-a502044a9949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:58.524094 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-909e2718-2421-48d1-915c-a502044a9949 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:58.525278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-909e2718-2421-48d1-915c-a502044a9949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:58.569927 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:58.606880 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 529/1053] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:58.631210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07ca43-af31-435f-bf23-b30fbdfed630 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:58.744391 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07ca43-af31-435f-bf23-b30fbdfed630 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:58.773499 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:58.782382 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d6f41b-a026-406b-98d1-e739f3ab40a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:58.783209 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6f41b-a026-406b-98d1-e739f3ab40a8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:58.783492 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6f41b-a026-406b-98d1-e739f3ab40a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:58.783765 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6f41b-a026-406b-98d1-e739f3ab40a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:58.795673 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d6f41b-a026-406b-98d1-e739f3ab40a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:46:58.803275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e9d6f41b-a026-406b-98d1-e739f3ab40a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:46:58.803275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:46:58.803275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:46:58.803275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:46:58.803275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:46:58.803275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:46:58.803275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:46:58.803275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:46:58.803275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:46:58.803275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:46:58.803275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:46:58.803275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:46:58.803275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:46:58.803275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:46:58.803275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:46:58.803275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:46:58.804219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:46:58.804219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:46:58.804219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:46:58.804219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:46:58.804219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:46:58.804219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:46:58.804219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:46:58.804219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:46:58.804219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:46:58.804219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:46:58.804219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:46:58.804219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:46:58.804219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:46:58.804219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:46:58.804219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:46:58.804219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:46:58.805177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:46:58.805177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:46:58.805177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:46:58.805177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:46:58.805177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:46:58.805177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:46:58.805177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:46:58.805177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:46:58.805177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:46:58.805177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:46:58.805177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:46:58.805177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:46:58.805177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 525/1054] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:46:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:46:58.811813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e2718-2421-48d1-915c-a502044a9949 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:58.914516 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e2718-2421-48d1-915c-a502044a9949 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:58.953257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298f907-9af6-45fc-aba6-c23fe21ff097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:58.953943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298f907-9af6-45fc-aba6-c23fe21ff097 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:58.954227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298f907-9af6-45fc-aba6-c23fe21ff097 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:58.954507 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298f907-9af6-45fc-aba6-c23fe21ff097 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:58.955364 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298f907-9af6-45fc-aba6-c23fe21ff097 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:58.955678 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298f907-9af6-45fc-aba6-c23fe21ff097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:58.957723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298f907-9af6-45fc-aba6-c23fe21ff097 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:58.974353 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 530/1055] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:46:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:46:58.975807 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d6f41b-a026-406b-98d1-e739f3ab40a8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:59.078223 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d6f41b-a026-406b-98d1-e739f3ab40a8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:59.112977 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f17fd-92da-4bd9-970e-824e00bfe139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:59.113925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f17fd-92da-4bd9-970e-824e00bfe139 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:59.114209 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f17fd-92da-4bd9-970e-824e00bfe139 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:59.114552 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f17fd-92da-4bd9-970e-824e00bfe139 None admin] PATH_INFO: `/v3/users/ec3019d5b2cc40e58aad2dad1bc94a3c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:59.129860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f17fd-92da-4bd9-970e-824e00bfe139 None admin] RBAC: Authorizing `identity:get_user(user_id=ec3019d5b2cc40e58aad2dad1bc94a3c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:59.132457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f17fd-92da-4bd9-970e-824e00bfe139 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:59.138772 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 526/1056] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:46:58 2022] GET /identity/v3/users/ec3019d5b2cc40e58aad2dad1bc94a3c => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:59.139854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298f907-9af6-45fc-aba6-c23fe21ff097 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:59.175639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:59.243861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298f907-9af6-45fc-aba6-c23fe21ff097 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:59.278334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7fb668-15f8-4813-8b50-48e670579c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:59.279015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7fb668-15f8-4813-8b50-48e670579c45 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:59.279301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7fb668-15f8-4813-8b50-48e670579c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:59.279579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7fb668-15f8-4813-8b50-48e670579c45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:59.351077 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7fb668-15f8-4813-8b50-48e670579c45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:59.352449 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7fb668-15f8-4813-8b50-48e670579c45 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:59.385805 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:59.463563 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 531/1057] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:46:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:46:59.464788 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f17fd-92da-4bd9-970e-824e00bfe139 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:59.515167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:59.572934 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f17fd-92da-4bd9-970e-824e00bfe139 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:59.608608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bcc00-eb72-4c5f-ab0c-6c10888cc3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:59.609398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bcc00-eb72-4c5f-ab0c-6c10888cc3bb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:59.609680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bcc00-eb72-4c5f-ab0c-6c10888cc3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:59.609953 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bcc00-eb72-4c5f-ab0c-6c10888cc3bb None admin] PATH_INFO: `/v3/projects/7a09ae933f9e40c1ab95266084ddff97` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:59.611154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4bcc00-eb72-4c5f-ab0c-6c10888cc3bb None admin] RBAC: Authorizing `identity:get_project(project_id=7a09ae933f9e40c1ab95266084ddff97)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:59.612583 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4bcc00-eb72-4c5f-ab0c-6c10888cc3bb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:59.620162 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 527/1058] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:46:59 2022] GET /identity/v3/projects/7a09ae933f9e40c1ab95266084ddff97 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:59.621772 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7fb668-15f8-4813-8b50-48e670579c45 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:59.727493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7fb668-15f8-4813-8b50-48e670579c45 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:59.767382 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c775ce-046d-406a-bead-3847d3070e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:59.768041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c775ce-046d-406a-bead-3847d3070e7e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:59.768324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c775ce-046d-406a-bead-3847d3070e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:59.768603 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c775ce-046d-406a-bead-3847d3070e7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:59.769434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c775ce-046d-406a-bead-3847d3070e7e None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=7a09ae933f9e40c1ab95266084ddff97, user.id=ec3019d5b2cc40e58aad2dad1bc94a3c) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:46:59.769813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c775ce-046d-406a-bead-3847d3070e7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:59.771938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c775ce-046d-406a-bead-3847d3070e7e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:59.773883 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:59.786878 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 532/1059] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:46:59 2022] GET /identity/v3/role_assignments?user.id=ec3019d5b2cc40e58aad2dad1bc94a3c&scope.project.id=7a09ae933f9e40c1ab95266084ddff97&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:46:59.800590 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df87fb-62dd-46ba-a744-7906173296b5 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:59.801469 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df87fb-62dd-46ba-a744-7906173296b5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:59.801725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df87fb-62dd-46ba-a744-7906173296b5 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:59.803394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 528/1060] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:46:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:46:59.809469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792ce60-a5fe-4773-babe-43f4a3f7c08e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:59.809763 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792ce60-a5fe-4773-babe-43f4a3f7c08e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:59.810027 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792ce60-a5fe-4773-babe-43f4a3f7c08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:59.828666 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4792ce60-a5fe-4773-babe-43f4a3f7c08e None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:46:59.829869 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4792ce60-a5fe-4773-babe-43f4a3f7c08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:59.910772 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 533/1061] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:46:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:46:59.932088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df87fb-62dd-46ba-a744-7906173296b5 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:00.041752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df87fb-62dd-46ba-a744-7906173296b5 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:00.057070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:00.079293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc5beb-5f91-4c2a-ac5c-ca1f5dc19f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:00.080162 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc5beb-5f91-4c2a-ac5c-ca1f5dc19f16 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:00.080445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc5beb-5f91-4c2a-ac5c-ca1f5dc19f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:00.080761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc5beb-5f91-4c2a-ac5c-ca1f5dc19f16 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:00.091511 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc5beb-5f91-4c2a-ac5c-ca1f5dc19f16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:47:00.100102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-44cc5beb-5f91-4c2a-ac5c-ca1f5dc19f16 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:47:00.100102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:00.100102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:00.100102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:00.100102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:00.100102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:00.100102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:00.100102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:00.100102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:00.100102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:00.100102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:00.100102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:00.100102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:47:00.100102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:47:00.100102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:47:00.100102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:47:00.101057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:47:00.101057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:47:00.101057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:47:00.101057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:47:00.101057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:00.101057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:00.101057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:00.101057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:00.101057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:00.101057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:00.101057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:00.101057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:00.101057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:00.101057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:00.101057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:00.101057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:00.107052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:00.107052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:00.107052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:00.107052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:00.107052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:47:00.107052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:47:00.107052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:47:00.107052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:47:00.107052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:47:00.107052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:47:00.107052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:47:00.107052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:47:00.107052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 529/1062] 10.222.0.70 () {56 vars in 1120 bytes} [Thu Jun 16 18:46:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:47:00.109688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792ce60-a5fe-4773-babe-43f4a3f7c08e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:00.216355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792ce60-a5fe-4773-babe-43f4a3f7c08e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:00.244637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:00.253441 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763ae4e-dd81-45d4-9d08-9378fe86d57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:00.254861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763ae4e-dd81-45d4-9d08-9378fe86d57f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:00.255387 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763ae4e-dd81-45d4-9d08-9378fe86d57f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:00.255387 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763ae4e-dd81-45d4-9d08-9378fe86d57f None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:00.256137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763ae4e-dd81-45d4-9d08-9378fe86d57f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:00.256445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763ae4e-dd81-45d4-9d08-9378fe86d57f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:00.257814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763ae4e-dd81-45d4-9d08-9378fe86d57f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:00.271749 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 534/1063] 10.222.0.70 () {56 vars in 1115 bytes} [Thu Jun 16 18:47:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:47:00.273009 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc5beb-5f91-4c2a-ac5c-ca1f5dc19f16 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:00.365569 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caca5a1-c79e-460e-a38f-ca72191ae522 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:00.365901 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caca5a1-c79e-460e-a38f-ca72191ae522 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:00.366155 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caca5a1-c79e-460e-a38f-ca72191ae522 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:00.367826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 535/1064] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:47:00.372631 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac3b01-b68c-4a6a-9a53-efa9dcfc09b9 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:00.372922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac3b01-b68c-4a6a-9a53-efa9dcfc09b9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:00.373203 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac3b01-b68c-4a6a-9a53-efa9dcfc09b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:00.387354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc5beb-5f91-4c2a-ac5c-ca1f5dc19f16 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:00.390306 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-08ac3b01-b68c-4a6a-9a53-efa9dcfc09b9 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:00.391531 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-08ac3b01-b68c-4a6a-9a53-efa9dcfc09b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:00.426605 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c1092-d612-44b4-b80d-f39ea33dcb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:00.427438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c1092-d612-44b4-b80d-f39ea33dcb96 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:00.427727 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c1092-d612-44b4-b80d-f39ea33dcb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:00.428008 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c1092-d612-44b4-b80d-f39ea33dcb96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:00.439334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c1092-d612-44b4-b80d-f39ea33dcb96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:00.440775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c1092-d612-44b4-b80d-f39ea33dcb96 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:00.458140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-089c1092-d612-44b4-b80d-f39ea33dcb96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:47:00.458140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:00.458140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:00.458140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:00.458140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:00.458140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:00.458140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:00.458140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:00.458140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:00.458140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:00.458140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:00.458140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:00.458140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:47:00.458140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:47:00.458140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:47:00.459105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:47:00.459105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:00.459105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:00.459105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:00.459105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:00.459105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:00.459105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:00.459105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:00.459105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:00.459105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:00.459105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:00.459105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:00.459105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:00.459105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:00.459105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:00.459105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:00.459105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:00.460579 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:47:00.460579 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:47:00.460579 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:47:00.460579 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:47:00.460579 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 530/1065] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:47:00.464936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c1092-d612-44b4-b80d-f39ea33dcb96 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:00.469981 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 536/1066] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:47:00.494406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac3b01-b68c-4a6a-9a53-efa9dcfc09b9 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:00.495074 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:00.575099 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c1092-d612-44b4-b80d-f39ea33dcb96 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:00.607630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac3b01-b68c-4a6a-9a53-efa9dcfc09b9 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:00.611498 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd74b7-19a7-43f9-aed7-5b167f82ba11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:00.612191 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd74b7-19a7-43f9-aed7-5b167f82ba11 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:00.612498 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd74b7-19a7-43f9-aed7-5b167f82ba11 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:00.612798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd74b7-19a7-43f9-aed7-5b167f82ba11 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:00.613627 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd74b7-19a7-43f9-aed7-5b167f82ba11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:00.613940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd74b7-19a7-43f9-aed7-5b167f82ba11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:00.615628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd74b7-19a7-43f9-aed7-5b167f82ba11 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:00.628118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 531/1067] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:47:00.631553 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:00.634256 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd74b7-19a7-43f9-aed7-5b167f82ba11 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:00.651628 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425976cf-c5c7-4dd8-8c73-ace13fa6dfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:00.652286 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425976cf-c5c7-4dd8-8c73-ace13fa6dfda None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:00.652623 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425976cf-c5c7-4dd8-8c73-ace13fa6dfda None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:00.652900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425976cf-c5c7-4dd8-8c73-ace13fa6dfda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:00.725615 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:00.728050 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425976cf-c5c7-4dd8-8c73-ace13fa6dfda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:00.729518 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425976cf-c5c7-4dd8-8c73-ace13fa6dfda None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:00.744898 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd74b7-19a7-43f9-aed7-5b167f82ba11 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:00.783755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d4c0e-af9f-4d59-b4b7-4e6e5ca2c323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:00.784669 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d4c0e-af9f-4d59-b4b7-4e6e5ca2c323 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:00.785086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d4c0e-af9f-4d59-b4b7-4e6e5ca2c323 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:00.785403 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d4c0e-af9f-4d59-b4b7-4e6e5ca2c323 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:00.793316 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d4c0e-af9f-4d59-b4b7-4e6e5ca2c323 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:00.794889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d4c0e-af9f-4d59-b4b7-4e6e5ca2c323 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:00.810639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-1f2d4c0e-af9f-4d59-b4b7-4e6e5ca2c323 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:47:00.810639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:00.810639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:00.810639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:00.810639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:00.810639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:00.810639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:00.810639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:00.810639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:00.810639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:00.810639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:00.810639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:00.810639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:47:00.810639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:47:00.810639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:47:00.810639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:00.811681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:00.812653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:00.812653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:00.812653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:47:00.812653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:47:00.812653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:47:00.812653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:47:00.812653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:47:00.812653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:47:00.812653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:47:00.812653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:47:00.812653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 532/1068] 10.222.0.70 () {56 vars in 1093 bytes} [Thu Jun 16 18:47:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:47:00.817808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d4c0e-af9f-4d59-b4b7-4e6e5ca2c323 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:00.836385 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 537/1069] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:47:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:47:00.842121 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425976cf-c5c7-4dd8-8c73-ace13fa6dfda None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:00.865023 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:00.950335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d4c0e-af9f-4d59-b4b7-4e6e5ca2c323 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:00.955681 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425976cf-c5c7-4dd8-8c73-ace13fa6dfda None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:00.987668 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192db15-2e98-48b4-916f-e768b4cfe515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:00.989101 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192db15-2e98-48b4-916f-e768b4cfe515 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:00.989537 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192db15-2e98-48b4-916f-e768b4cfe515 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:00.990006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192db15-2e98-48b4-916f-e768b4cfe515 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:00.991053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192db15-2e98-48b4-916f-e768b4cfe515 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:00.991540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192db15-2e98-48b4-916f-e768b4cfe515 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:00.992924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c77a1d-2437-4158-ab73-992d8e01687e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:00.993061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192db15-2e98-48b4-916f-e768b4cfe515 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:00.993719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c77a1d-2437-4158-ab73-992d8e01687e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:00.994002 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c77a1d-2437-4158-ab73-992d8e01687e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:00.994279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c77a1d-2437-4158-ab73-992d8e01687e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:01.006079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c77a1d-2437-4158-ab73-992d8e01687e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:01.007425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c77a1d-2437-4158-ab73-992d8e01687e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:01.017191 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 533/1070] 10.222.0.70 () {56 vars in 1133 bytes} [Thu Jun 16 18:47:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:01.018166 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192db15-2e98-48b4-916f-e768b4cfe515 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:01.021981 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:01.030602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-15c77a1d-2437-4158-ab73-992d8e01687e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:47:01.030602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:01.030602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:01.030602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:01.030602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:01.030602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:01.030602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:01.030602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:01.030602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:01.030602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:01.030602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:01.030602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:01.030602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:47:01.030602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:47:01.030602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:47:01.031509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:47:01.031509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:01.031509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:01.031509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:01.031509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:01.031509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:01.031509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:01.031509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:01.031509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:01.031509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:01.031509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:01.031509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:01.031509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:01.031509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:01.031509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:01.031509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:01.031509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:01.032426 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:47:01.032426 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:47:01.032426 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:47:01.032426 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:47:01.033051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 538/1071] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:47:01.039636 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c77a1d-2437-4158-ab73-992d8e01687e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:01.116973 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:01.129452 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192db15-2e98-48b4-916f-e768b4cfe515 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:01.154054 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c77a1d-2437-4158-ab73-992d8e01687e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:01.173154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17019d27-6f32-47b5-ac8a-57fe64ef55f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:01.173953 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17019d27-6f32-47b5-ac8a-57fe64ef55f5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:01.174236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17019d27-6f32-47b5-ac8a-57fe64ef55f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:01.174686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17019d27-6f32-47b5-ac8a-57fe64ef55f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:01.186786 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17019d27-6f32-47b5-ac8a-57fe64ef55f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:01.188224 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17019d27-6f32-47b5-ac8a-57fe64ef55f5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:01.194814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fd2fe-e25f-4af1-844f-a4cecd6cf3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:01.195542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fd2fe-e25f-4af1-844f-a4cecd6cf3fb None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:01.195828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fd2fe-e25f-4af1-844f-a4cecd6cf3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:01.196107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fd2fe-e25f-4af1-844f-a4cecd6cf3fb None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:01.196830 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fd2fe-e25f-4af1-844f-a4cecd6cf3fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:01.197137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fd2fe-e25f-4af1-844f-a4cecd6cf3fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:01.198753 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fd2fe-e25f-4af1-844f-a4cecd6cf3fb None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:01.211978 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 539/1072] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:47:01.212476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-17019d27-6f32-47b5-ac8a-57fe64ef55f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:47:01.212476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:01.212476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:01.212476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:01.212476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:01.212476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:01.212476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:01.212476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:01.212476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:01.212476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:01.212476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:01.212476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:01.212476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:47:01.212476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:47:01.212476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:47:01.213631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:47:01.213631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:01.213631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:01.213631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:01.213631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:01.213631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:01.213631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:01.213631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:01.213631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:01.213631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:01.213631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:01.213631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:01.213631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:01.213631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:01.213631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:01.213631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:01.213631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:01.215532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:47:01.215532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:47:01.215532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:47:01.215532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:47:01.218684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 534/1073] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:47:01.219844 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fd2fe-e25f-4af1-844f-a4cecd6cf3fb None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:01.339828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fd2fe-e25f-4af1-844f-a4cecd6cf3fb None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:01.370866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:01.378097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616021db-90cf-48bd-baff-40271a83860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:01.379659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616021db-90cf-48bd-baff-40271a83860a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:01.379872 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616021db-90cf-48bd-baff-40271a83860a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:01.380156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616021db-90cf-48bd-baff-40271a83860a None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:01.381128 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616021db-90cf-48bd-baff-40271a83860a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:01.381323 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616021db-90cf-48bd-baff-40271a83860a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:01.382730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616021db-90cf-48bd-baff-40271a83860a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:01.396571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 540/1074] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:01.402143 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17019d27-6f32-47b5-ac8a-57fe64ef55f5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:01.469098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:01.517337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17019d27-6f32-47b5-ac8a-57fe64ef55f5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:01.552759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3917d53-c0b3-4c05-bebe-7537756cfbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:01.553554 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3917d53-c0b3-4c05-bebe-7537756cfbad None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:01.553837 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3917d53-c0b3-4c05-bebe-7537756cfbad None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:01.554112 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3917d53-c0b3-4c05-bebe-7537756cfbad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:01.560867 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3917d53-c0b3-4c05-bebe-7537756cfbad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:01.562303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3917d53-c0b3-4c05-bebe-7537756cfbad None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:01.577827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c3917d53-c0b3-4c05-bebe-7537756cfbad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:47:01.577827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:01.577827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:01.577827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:01.577827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:01.577827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:01.577827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:01.577827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:01.577827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:01.577827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:01.577827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:01.577827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:01.577827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:47:01.577827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:47:01.577827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:47:01.579302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:47:01.579302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:01.579302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:01.579302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:01.579302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:01.579302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:01.579302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:01.579302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:01.579302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:01.579302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:01.579302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:01.579302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:01.579302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:01.579302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:01.579302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:01.579302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:01.579302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:01.585373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:47:01.585373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:47:01.585373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:47:01.585373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:47:01.585373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:47:01.585373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:47:01.585373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:47:01.585373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:47:01.585373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 535/1075] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:47:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:47:01.586214 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616021db-90cf-48bd-baff-40271a83860a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:01.687513 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616021db-90cf-48bd-baff-40271a83860a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:01.722052 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9ad9a-3c7d-4c9e-a10e-0a8d7e293bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:01.722749 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9ad9a-3c7d-4c9e-a10e-0a8d7e293bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:01.723032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9ad9a-3c7d-4c9e-a10e-0a8d7e293bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:01.723311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9ad9a-3c7d-4c9e-a10e-0a8d7e293bb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:01.724313 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9ad9a-3c7d-4c9e-a10e-0a8d7e293bb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:01.724635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9ad9a-3c7d-4c9e-a10e-0a8d7e293bb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:01.726067 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9ad9a-3c7d-4c9e-a10e-0a8d7e293bb6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:01.738024 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 541/1076] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:47:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:01.744707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3917d53-c0b3-4c05-bebe-7537756cfbad None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:01.846772 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3917d53-c0b3-4c05-bebe-7537756cfbad None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:01.865470 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:01.885592 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d2ae7-e125-4bc1-9249-c29d231cfeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:01.886490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d2ae7-e125-4bc1-9249-c29d231cfeb1 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:01.886810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d2ae7-e125-4bc1-9249-c29d231cfeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:01.887088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d2ae7-e125-4bc1-9249-c29d231cfeb1 None admin] PATH_INFO: `/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/8342da838fe4462b9de2817ad3ac4ff7/roles/ed8734ab73344d1983b1cdaa7fa3238a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:01.889296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d2ae7-e125-4bc1-9249-c29d231cfeb1 None admin] RBAC: Authorizing `identity:create_grant(project_id=34185125e75d4aad8ad5382ab22ee8a2, user_id=8342da838fe4462b9de2817ad3ac4ff7, role_id=ed8734ab73344d1983b1cdaa7fa3238a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:01.890749 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d2ae7-e125-4bc1-9249-c29d231cfeb1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:01.904454 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 536/1077] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:47:01 2022] PUT /identity/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/8342da838fe4462b9de2817ad3ac4ff7/roles/ed8734ab73344d1983b1cdaa7fa3238a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:47:01.917082 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280e8b4-ca11-4a59-805d-a4823f3970d1 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:01.917374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280e8b4-ca11-4a59-805d-a4823f3970d1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:01.917651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280e8b4-ca11-4a59-805d-a4823f3970d1 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:01.919206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 542/1078] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:01.927754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58902c81-0962-45cb-b520-7a2ffa2187ef None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:01.928055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58902c81-0962-45cb-b520-7a2ffa2187ef None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:01.928317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58902c81-0962-45cb-b520-7a2ffa2187ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:01.947284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-58902c81-0962-45cb-b520-7a2ffa2187ef None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:01.948426 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-58902c81-0962-45cb-b520-7a2ffa2187ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:02.023280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 537/1079] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:47:02.047366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280e8b4-ca11-4a59-805d-a4823f3970d1 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:02.106234 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:02.155094 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280e8b4-ca11-4a59-805d-a4823f3970d1 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:02.188839 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8e299-a42d-44af-bbbc-91f513226fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:02.189683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8e299-a42d-44af-bbbc-91f513226fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:02.189967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8e299-a42d-44af-bbbc-91f513226fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:02.190243 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8e299-a42d-44af-bbbc-91f513226fe4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:02.201072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8e299-a42d-44af-bbbc-91f513226fe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:47:02.209272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-1dd8e299-a42d-44af-bbbc-91f513226fe4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:47:02.209272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:02.209272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:02.209272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:02.209272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:02.209272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:02.209272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:02.209272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:02.209272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:02.209272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:02.209272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:02.209272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:02.209272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:47:02.209272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:47:02.209272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:47:02.209272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:47:02.210429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:47:02.210429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:47:02.210429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:47:02.210429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:47:02.210429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:02.210429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:02.210429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:02.210429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:02.210429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:02.210429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:02.210429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:02.210429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:02.210429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:02.210429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:02.210429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:02.210429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:02.211674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:02.211674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:02.211674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:02.211674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:02.211674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:47:02.211674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:47:02.211674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:47:02.211674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:47:02.211674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:47:02.211674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:47:02.211674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:47:02.211674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:47:02.211674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 543/1080] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:47:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:47:02.216758 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58902c81-0962-45cb-b520-7a2ffa2187ef None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:02.318589 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58902c81-0962-45cb-b520-7a2ffa2187ef None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:02.325982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:02.354082 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b81aa-2119-44ce-82de-3da5dcc35c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:02.354784 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b81aa-2119-44ce-82de-3da5dcc35c8e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:02.355071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b81aa-2119-44ce-82de-3da5dcc35c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:02.355348 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b81aa-2119-44ce-82de-3da5dcc35c8e None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:02.356091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b81aa-2119-44ce-82de-3da5dcc35c8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:02.356399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b81aa-2119-44ce-82de-3da5dcc35c8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:02.357787 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b81aa-2119-44ce-82de-3da5dcc35c8e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:02.367209 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 538/1081] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:47:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:02.372863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8e299-a42d-44af-bbbc-91f513226fe4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:02.473923 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8e299-a42d-44af-bbbc-91f513226fe4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:02.487741 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:02.510524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33818ade-e7b6-4dc2-8a99-63b2432e3a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:02.511410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33818ade-e7b6-4dc2-8a99-63b2432e3a60 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:02.511692 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33818ade-e7b6-4dc2-8a99-63b2432e3a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:02.511962 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33818ade-e7b6-4dc2-8a99-63b2432e3a60 None admin] PATH_INFO: `/v3/users/ec3019d5b2cc40e58aad2dad1bc94a3c` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:02.513143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33818ade-e7b6-4dc2-8a99-63b2432e3a60 None admin] RBAC: Authorizing `identity:get_user(user_id=ec3019d5b2cc40e58aad2dad1bc94a3c)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:02.514561 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33818ade-e7b6-4dc2-8a99-63b2432e3a60 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:02.516206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 544/1082] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:47:02 2022] GET /identity/v3/users/ec3019d5b2cc40e58aad2dad1bc94a3c => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:02.522814 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b81aa-2119-44ce-82de-3da5dcc35c8e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:02.627807 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b81aa-2119-44ce-82de-3da5dcc35c8e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:02.662126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a783fd7-c181-49ec-be3a-fc849b8c77d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:02.663035 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a783fd7-c181-49ec-be3a-fc849b8c77d9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:02.663321 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a783fd7-c181-49ec-be3a-fc849b8c77d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:02.663598 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a783fd7-c181-49ec-be3a-fc849b8c77d9 None admin] PATH_INFO: `/v3/projects/7a09ae933f9e40c1ab95266084ddff97` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:02.664789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a783fd7-c181-49ec-be3a-fc849b8c77d9 None admin] RBAC: Authorizing `identity:get_project(project_id=7a09ae933f9e40c1ab95266084ddff97)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:02.666215 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a783fd7-c181-49ec-be3a-fc849b8c77d9 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:02.668113 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 539/1083] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:47:02 2022] GET /identity/v3/projects/7a09ae933f9e40c1ab95266084ddff97 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:02.673931 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33818ade-e7b6-4dc2-8a99-63b2432e3a60 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:02.774806 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33818ade-e7b6-4dc2-8a99-63b2432e3a60 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:02.809125 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d806510-6fcf-4450-a920-d67ab0360966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:02.810006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d806510-6fcf-4450-a920-d67ab0360966 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:02.810288 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d806510-6fcf-4450-a920-d67ab0360966 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:02.810564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d806510-6fcf-4450-a920-d67ab0360966 None admin] PATH_INFO: `/v3/projects/7a09ae933f9e40c1ab95266084ddff97/users/ec3019d5b2cc40e58aad2dad1bc94a3c/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:02.812529 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d806510-6fcf-4450-a920-d67ab0360966 None admin] RBAC: Authorizing `identity:create_grant(project_id=7a09ae933f9e40c1ab95266084ddff97, user_id=ec3019d5b2cc40e58aad2dad1bc94a3c, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:02.813948 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d806510-6fcf-4450-a920-d67ab0360966 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:02.828359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 545/1084] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:47:02 2022] PUT /identity/v3/projects/7a09ae933f9e40c1ab95266084ddff97/users/ec3019d5b2cc40e58aad2dad1bc94a3c/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:47:04.069302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c493131-a250-40b3-9b8c-e0181ff20671 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:04.069859 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c493131-a250-40b3-9b8c-e0181ff20671 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:04.069859 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c493131-a250-40b3-9b8c-e0181ff20671 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:04.071269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 540/1085] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:04.076510 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75c22b-72f8-4c78-a92e-c56e5423154a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:04.076934 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75c22b-72f8-4c78-a92e-c56e5423154a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:04.077189 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75c22b-72f8-4c78-a92e-c56e5423154a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:04.094238 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4a75c22b-72f8-4c78-a92e-c56e5423154a None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:04.095435 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4a75c22b-72f8-4c78-a92e-c56e5423154a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:04.128511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:04.163824 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 546/1086] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:47:04.187308 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c493131-a250-40b3-9b8c-e0181ff20671 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:04.224664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:04.290063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c493131-a250-40b3-9b8c-e0181ff20671 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:04.324197 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59a9c3-ef01-4f77-995c-71985ab72008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:04.325093 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59a9c3-ef01-4f77-995c-71985ab72008 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:04.325375 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59a9c3-ef01-4f77-995c-71985ab72008 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:04.325647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59a9c3-ef01-4f77-995c-71985ab72008 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:04.337383 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59a9c3-ef01-4f77-995c-71985ab72008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:47:04.345866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-cb59a9c3-ef01-4f77-995c-71985ab72008 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:47:04.345866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:04.345866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:04.345866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:04.345866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:04.345866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:04.345866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:04.345866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:04.345866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:04.345866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:04.345866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:04.345866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:04.345866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:47:04.345866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:47:04.345866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:47:04.345866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:47:04.353306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:47:04.353306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:47:04.353306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:47:04.353306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:47:04.353306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:04.353306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:04.353306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:04.353306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:04.353306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:04.353306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:04.353306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:04.353306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:04.353306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:04.353306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:04.353306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:04.353306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:04.354428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:04.354428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:04.354428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:04.354428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:04.354428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:47:04.354428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:47:04.354428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:47:04.354428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:47:04.354428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:47:04.354428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:47:04.354428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:47:04.354428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:47:04.354428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 541/1087] 10.222.0.70 () {56 vars in 1120 bytes} [Thu Jun 16 18:47:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:47:04.355622 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75c22b-72f8-4c78-a92e-c56e5423154a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:04.454742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75c22b-72f8-4c78-a92e-c56e5423154a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:04.487748 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b632784-6055-4b37-a2e7-47793bd7d2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:04.488863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b632784-6055-4b37-a2e7-47793bd7d2ba None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:04.489143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b632784-6055-4b37-a2e7-47793bd7d2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:04.489560 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b632784-6055-4b37-a2e7-47793bd7d2ba None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:04.490295 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b632784-6055-4b37-a2e7-47793bd7d2ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:04.490630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b632784-6055-4b37-a2e7-47793bd7d2ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:04.491990 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b632784-6055-4b37-a2e7-47793bd7d2ba None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:04.500030 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 547/1088] 10.222.0.70 () {56 vars in 1115 bytes} [Thu Jun 16 18:47:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:04.506495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59a9c3-ef01-4f77-995c-71985ab72008 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:04.601131 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:04.607984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59a9c3-ef01-4f77-995c-71985ab72008 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:04.643028 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3fc13-477e-4514-a0d5-2c85d22d36c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:04.643817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3fc13-477e-4514-a0d5-2c85d22d36c0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:04.644100 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3fc13-477e-4514-a0d5-2c85d22d36c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:04.644435 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3fc13-477e-4514-a0d5-2c85d22d36c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:04.654758 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3fc13-477e-4514-a0d5-2c85d22d36c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:04.656188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3fc13-477e-4514-a0d5-2c85d22d36c0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:04.673518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-d0a3fc13-477e-4514-a0d5-2c85d22d36c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:47:04.673518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:04.673518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:04.673518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:04.673518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:04.673518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:04.673518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:04.673518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:04.673518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:04.673518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:04.673518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:04.673518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:04.673518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:47:04.673518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:47:04.673518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:47:04.674705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:47:04.674705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:04.674705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:04.674705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:04.674705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:04.674705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:04.674705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:04.674705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:04.674705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:04.674705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:04.674705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:04.674705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:04.674705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:04.674705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:04.674705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:04.674705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:04.674705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:04.676164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:47:04.676164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:47:04.676164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:47:04.676164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:47:04.676164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 542/1089] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:47:04.681345 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b632784-6055-4b37-a2e7-47793bd7d2ba None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:04.720534 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:04.792386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b632784-6055-4b37-a2e7-47793bd7d2ba None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:04.825560 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3e8f0-b99a-482b-93b8-0e33c25675c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:04.826217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3e8f0-b99a-482b-93b8-0e33c25675c4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:04.826498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3e8f0-b99a-482b-93b8-0e33c25675c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:04.826801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3e8f0-b99a-482b-93b8-0e33c25675c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:04.827514 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3e8f0-b99a-482b-93b8-0e33c25675c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:04.827821 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3e8f0-b99a-482b-93b8-0e33c25675c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:04.829177 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3e8f0-b99a-482b-93b8-0e33c25675c4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:04.841429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 548/1090] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:04.847078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3fc13-477e-4514-a0d5-2c85d22d36c0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:04.952432 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3fc13-477e-4514-a0d5-2c85d22d36c0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:04.986280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4e09c-3d2c-4005-afdd-0402e07f12e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:04.987101 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4e09c-3d2c-4005-afdd-0402e07f12e2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:04.987379 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4e09c-3d2c-4005-afdd-0402e07f12e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:04.987565 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab7023-3b3f-4b34-bc42-f6352dec830e None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:04.987682 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4e09c-3d2c-4005-afdd-0402e07f12e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:04.987859 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab7023-3b3f-4b34-bc42-f6352dec830e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:04.988167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab7023-3b3f-4b34-bc42-f6352dec830e None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:04.989706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 549/1091] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:47:04.994820 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19d1b4-e17a-4aed-8e52-62cd3bf8a1a6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:04.995181 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19d1b4-e17a-4aed-8e52-62cd3bf8a1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:04.995466 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19d1b4-e17a-4aed-8e52-62cd3bf8a1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:04.995726 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4e09c-3d2c-4005-afdd-0402e07f12e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:04.997154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4e09c-3d2c-4005-afdd-0402e07f12e2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:05.011614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-56a4e09c-3d2c-4005-afdd-0402e07f12e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:47:05.011614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:05.011614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:05.011614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:05.011614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:05.011614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:05.011614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:05.011614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:05.011614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:05.011614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:05.011614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:05.011614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:05.011614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:47:05.011614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:47:05.011614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:47:05.011614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:05.012798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:05.013778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:05.013778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:05.013778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:47:05.013778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:47:05.013778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:47:05.013778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:47:05.013778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:47:05.013778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:47:05.013778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:47:05.013778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:47:05.014341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-9a19d1b4-e17a-4aed-8e52-62cd3bf8a1a6 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:05.014416 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 543/1092] 10.222.0.70 () {56 vars in 1093 bytes} [Thu Jun 16 18:47:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:47:05.014491 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9a19d1b4-e17a-4aed-8e52-62cd3bf8a1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:05.018680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4e09c-3d2c-4005-afdd-0402e07f12e2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:05.030070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:05.090202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 550/1093] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:47:05.113895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19d1b4-e17a-4aed-8e52-62cd3bf8a1a6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:05.135642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4e09c-3d2c-4005-afdd-0402e07f12e2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:05.138080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:05.173323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e4f18-9c62-4c06-80cd-bf2692c444a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:05.173988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e4f18-9c62-4c06-80cd-bf2692c444a3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:05.174268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e4f18-9c62-4c06-80cd-bf2692c444a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:05.174539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e4f18-9c62-4c06-80cd-bf2692c444a3 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:05.175392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e4f18-9c62-4c06-80cd-bf2692c444a3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:05.176220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e4f18-9c62-4c06-80cd-bf2692c444a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:05.177752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e4f18-9c62-4c06-80cd-bf2692c444a3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:05.195002 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 544/1094] 10.222.0.70 () {56 vars in 1133 bytes} [Thu Jun 16 18:47:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:47:05.200529 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e4f18-9c62-4c06-80cd-bf2692c444a3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:05.217693 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19d1b4-e17a-4aed-8e52-62cd3bf8a1a6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:05.253238 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa416c-e56e-45f0-aabc-97997ae32eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:05.254337 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa416c-e56e-45f0-aabc-97997ae32eff None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:05.254814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa416c-e56e-45f0-aabc-97997ae32eff None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:05.255094 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa416c-e56e-45f0-aabc-97997ae32eff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:05.265316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa416c-e56e-45f0-aabc-97997ae32eff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:47:05.267753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:05.274015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-a0fa416c-e56e-45f0-aabc-97997ae32eff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:47:05.274015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:05.274015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:05.274015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:05.274015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:05.274015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:05.274015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:05.274015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:05.274015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:05.274015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:05.274015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:05.274015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:05.274015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:47:05.274015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:47:05.274015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:47:05.274015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:47:05.275021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:47:05.275021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:47:05.275021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:47:05.275021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:47:05.275021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:05.275021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:05.275021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:05.275021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:05.275021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:05.275021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:05.275021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:05.275021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:05.275021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:05.275021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:05.275021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:05.275021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:05.275879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:05.275879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:05.275879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:05.275879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:05.275879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:47:05.275879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:47:05.275879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:47:05.275879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:47:05.275879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:47:05.275879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:47:05.275879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:47:05.275879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:47:05.275879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 551/1095] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:47:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:47:05.281034 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa416c-e56e-45f0-aabc-97997ae32eff None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:05.304273 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e4f18-9c62-4c06-80cd-bf2692c444a3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:05.338972 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09507f-4668-4e11-9e06-fdac25edbb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:05.339618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09507f-4668-4e11-9e06-fdac25edbb2f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:05.339903 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09507f-4668-4e11-9e06-fdac25edbb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:05.340173 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09507f-4668-4e11-9e06-fdac25edbb2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:05.364028 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:05.382279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa416c-e56e-45f0-aabc-97997ae32eff None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:05.411455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf09507f-4668-4e11-9e06-fdac25edbb2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:05.412903 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf09507f-4668-4e11-9e06-fdac25edbb2f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:05.417882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3f5d9-4043-4c94-9cef-1e018100f7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:05.418544 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3f5d9-4043-4c94-9cef-1e018100f7fe None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:05.418861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3f5d9-4043-4c94-9cef-1e018100f7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:05.419138 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3f5d9-4043-4c94-9cef-1e018100f7fe None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:05.419878 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3f5d9-4043-4c94-9cef-1e018100f7fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:05.420187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3f5d9-4043-4c94-9cef-1e018100f7fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:05.421538 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3f5d9-4043-4c94-9cef-1e018100f7fe None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:05.430387 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 552/1096] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:47:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:47:05.436110 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3f5d9-4043-4c94-9cef-1e018100f7fe None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:05.479989 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:05.515676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 545/1097] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:47:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:47:05.522691 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09507f-4668-4e11-9e06-fdac25edbb2f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:05.537011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3f5d9-4043-4c94-9cef-1e018100f7fe None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:05.571295 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d30c90-239f-4031-8165-bc36d8f79ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:05.572199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d30c90-239f-4031-8165-bc36d8f79ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:05.572488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d30c90-239f-4031-8165-bc36d8f79ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:05.572766 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d30c90-239f-4031-8165-bc36d8f79ea9 None admin] PATH_INFO: `/v3/users/ec3019d5b2cc40e58aad2dad1bc94a3c` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:05.573945 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d30c90-239f-4031-8165-bc36d8f79ea9 None admin] RBAC: Authorizing `identity:get_user(user_id=ec3019d5b2cc40e58aad2dad1bc94a3c)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:05.575397 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d30c90-239f-4031-8165-bc36d8f79ea9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:05.577029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 553/1098] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:47:05 2022] GET /identity/v3/users/ec3019d5b2cc40e58aad2dad1bc94a3c => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:47:05.583277 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d30c90-239f-4031-8165-bc36d8f79ea9 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:05.598465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:05.621166 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09507f-4668-4e11-9e06-fdac25edbb2f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:05.655769 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83399c31-8d04-47e1-920b-83681b7ac758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:05.656555 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83399c31-8d04-47e1-920b-83681b7ac758 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:05.656837 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83399c31-8d04-47e1-920b-83681b7ac758 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:05.657110 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83399c31-8d04-47e1-920b-83681b7ac758 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:05.667371 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83399c31-8d04-47e1-920b-83681b7ac758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:05.668792 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83399c31-8d04-47e1-920b-83681b7ac758 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:05.683727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d30c90-239f-4031-8165-bc36d8f79ea9 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:05.686545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-83399c31-8d04-47e1-920b-83681b7ac758 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:47:05.686545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:05.686545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:05.686545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:05.686545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:05.686545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:05.686545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:05.686545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:05.686545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:05.686545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:05.686545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:05.686545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:05.686545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:47:05.686545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:47:05.686545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:47:05.687728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:47:05.687728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:05.687728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:05.687728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:05.687728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:05.687728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:05.687728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:05.687728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:05.687728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:05.687728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:05.687728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:05.687728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:05.687728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:05.687728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:05.687728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:05.687728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:05.687728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:05.688770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:47:05.688770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:47:05.688770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:47:05.688770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:47:05.688770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 546/1099] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:47:05.694473 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83399c31-8d04-47e1-920b-83681b7ac758 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:05.713154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:05.718261 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929bcb3-b198-4b0a-a37c-2b3594918c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:05.718937 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929bcb3-b198-4b0a-a37c-2b3594918c28 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:05.719223 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929bcb3-b198-4b0a-a37c-2b3594918c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:05.719494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929bcb3-b198-4b0a-a37c-2b3594918c28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:05.788229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929bcb3-b198-4b0a-a37c-2b3594918c28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:05.789906 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929bcb3-b198-4b0a-a37c-2b3594918c28 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:05.794027 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83399c31-8d04-47e1-920b-83681b7ac758 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:05.803837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:05.828751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c19704-4eaa-43d7-b068-adf66f316c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:05.829413 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c19704-4eaa-43d7-b068-adf66f316c3c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:05.829700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c19704-4eaa-43d7-b068-adf66f316c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:05.829978 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c19704-4eaa-43d7-b068-adf66f316c3c None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:05.830732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c19704-4eaa-43d7-b068-adf66f316c3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:05.831041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c19704-4eaa-43d7-b068-adf66f316c3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:05.835462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c19704-4eaa-43d7-b068-adf66f316c3c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:05.846557 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 547/1100] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:47:05.852700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c19704-4eaa-43d7-b068-adf66f316c3c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:05.892016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 554/1101] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:47:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:47:05.897995 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929bcb3-b198-4b0a-a37c-2b3594918c28 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:05.930167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:05.954618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c19704-4eaa-43d7-b068-adf66f316c3c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:05.989393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48bbc4-e8d1-4986-881a-5cd70e2b2c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:05.990187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48bbc4-e8d1-4986-881a-5cd70e2b2c96 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:05.990473 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48bbc4-e8d1-4986-881a-5cd70e2b2c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:05.990779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48bbc4-e8d1-4986-881a-5cd70e2b2c96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:05.997486 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48bbc4-e8d1-4986-881a-5cd70e2b2c96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:05.998950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48bbc4-e8d1-4986-881a-5cd70e2b2c96 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:06.003401 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929bcb3-b198-4b0a-a37c-2b3594918c28 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:06.013817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-cc48bbc4-e8d1-4986-881a-5cd70e2b2c96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:47:06.013817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:06.013817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:06.013817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:06.013817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:06.013817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:06.013817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:06.013817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:06.013817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:06.013817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:06.013817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:06.013817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:06.013817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:47:06.013817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:47:06.013817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:47:06.014900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:47:06.014900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:06.014900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:06.014900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:06.014900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:06.014900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:06.014900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:06.014900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:06.014900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:06.014900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:06.014900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:06.014900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:06.014900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:06.014900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:06.014900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:06.014900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:06.014900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:06.015923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:47:06.015923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:47:06.015923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:47:06.015923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:47:06.015923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:47:06.015923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:47:06.015923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:47:06.015923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:47:06.015923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 548/1102] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:47:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:47:06.018994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:06.021144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48bbc4-e8d1-4986-881a-5cd70e2b2c96 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:06.041825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409d64d-2307-4f25-bdb7-2fa4728fabaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:06.042773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409d64d-2307-4f25-bdb7-2fa4728fabaa None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:06.043086 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409d64d-2307-4f25-bdb7-2fa4728fabaa None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:06.043385 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409d64d-2307-4f25-bdb7-2fa4728fabaa None admin] PATH_INFO: `/v3/projects/7a09ae933f9e40c1ab95266084ddff97` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:06.044569 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6409d64d-2307-4f25-bdb7-2fa4728fabaa None admin] RBAC: Authorizing `identity:get_project(project_id=7a09ae933f9e40c1ab95266084ddff97)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:06.046069 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6409d64d-2307-4f25-bdb7-2fa4728fabaa None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:06.047912 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 555/1103] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:47:05 2022] GET /identity/v3/projects/7a09ae933f9e40c1ab95266084ddff97 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:47:06.054736 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409d64d-2307-4f25-bdb7-2fa4728fabaa None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:06.122887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48bbc4-e8d1-4986-881a-5cd70e2b2c96 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:06.154889 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409d64d-2307-4f25-bdb7-2fa4728fabaa None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:06.157405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19cf17-0a1c-4992-9a56-1072bc0ee8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:06.158062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19cf17-0a1c-4992-9a56-1072bc0ee8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:06.158346 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19cf17-0a1c-4992-9a56-1072bc0ee8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:06.158657 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19cf17-0a1c-4992-9a56-1072bc0ee8d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:06.159467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19cf17-0a1c-4992-9a56-1072bc0ee8d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:06.159802 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19cf17-0a1c-4992-9a56-1072bc0ee8d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:06.161780 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19cf17-0a1c-4992-9a56-1072bc0ee8d7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:06.168991 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:06.174738 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 549/1104] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:47:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:06.181276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19cf17-0a1c-4992-9a56-1072bc0ee8d7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:06.190587 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615d9b5-ff9b-4dfd-a396-965fd7c5a102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:06.191935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615d9b5-ff9b-4dfd-a396-965fd7c5a102 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:06.192243 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615d9b5-ff9b-4dfd-a396-965fd7c5a102 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:06.192484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615d9b5-ff9b-4dfd-a396-965fd7c5a102 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:06.193305 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615d9b5-ff9b-4dfd-a396-965fd7c5a102 None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=7a09ae933f9e40c1ab95266084ddff97, user.id=ec3019d5b2cc40e58aad2dad1bc94a3c) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:06.193609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615d9b5-ff9b-4dfd-a396-965fd7c5a102 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:06.195059 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615d9b5-ff9b-4dfd-a396-965fd7c5a102 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:06.200931 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 556/1105] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:47:06 2022] GET /identity/v3/role_assignments?user.id=ec3019d5b2cc40e58aad2dad1bc94a3c&scope.project.id=7a09ae933f9e40c1ab95266084ddff97&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:47:06.286881 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19cf17-0a1c-4992-9a56-1072bc0ee8d7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:06.323319 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22f2fa-c61e-415c-8ca7-0aba8b16e70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:06.323995 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22f2fa-c61e-415c-8ca7-0aba8b16e70f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:06.324282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22f2fa-c61e-415c-8ca7-0aba8b16e70f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:06.324563 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22f2fa-c61e-415c-8ca7-0aba8b16e70f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:06.325413 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22f2fa-c61e-415c-8ca7-0aba8b16e70f None admin] RBAC: Adding query filter params (role.id=ed8734ab73344d1983b1cdaa7fa3238a, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=8342da838fe4462b9de2817ad3ac4ff7) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:06.325731 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22f2fa-c61e-415c-8ca7-0aba8b16e70f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:06.327226 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22f2fa-c61e-415c-8ca7-0aba8b16e70f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:06.333834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 550/1106] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:47:06 2022] GET /identity/v3/role_assignments?user.id=8342da838fe4462b9de2817ad3ac4ff7&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=ed8734ab73344d1983b1cdaa7fa3238a => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:47:08.369033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d96e5f-4882-4ed5-8e9c-8a653c10b7ba None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:08.369550 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d96e5f-4882-4ed5-8e9c-8a653c10b7ba None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:08.369550 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d96e5f-4882-4ed5-8e9c-8a653c10b7ba None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:08.371017 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 557/1107] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:08.376120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a2b5c-cc76-4834-9d15-eee500cdefed None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:08.376548 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a2b5c-cc76-4834-9d15-eee500cdefed None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:08.376669 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a2b5c-cc76-4834-9d15-eee500cdefed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:08.393442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-012a2b5c-cc76-4834-9d15-eee500cdefed None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:08.395165 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-012a2b5c-cc76-4834-9d15-eee500cdefed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:08.449860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:08.461269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 551/1108] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:47:08.486762 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d96e5f-4882-4ed5-8e9c-8a653c10b7ba None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:08.532967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:08.587748 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d96e5f-4882-4ed5-8e9c-8a653c10b7ba None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:08.621048 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c59ad-bec2-43a7-8721-b37b5445ac7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:08.621773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c59ad-bec2-43a7-8721-b37b5445ac7e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:08.622060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c59ad-bec2-43a7-8721-b37b5445ac7e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:08.622333 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c59ad-bec2-43a7-8721-b37b5445ac7e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:08.623526 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c59ad-bec2-43a7-8721-b37b5445ac7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:08.624946 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c59ad-bec2-43a7-8721-b37b5445ac7e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:08.626547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 558/1109] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:47:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:08.633341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a2b5c-cc76-4834-9d15-eee500cdefed None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:08.733018 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a2b5c-cc76-4834-9d15-eee500cdefed None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:08.766859 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a7da5-77b1-4dfe-9461-a0945559ea9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:08.767530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a7da5-77b1-4dfe-9461-a0945559ea9c None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:08.767818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a7da5-77b1-4dfe-9461-a0945559ea9c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:08.768096 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a7da5-77b1-4dfe-9461-a0945559ea9c None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:08.768795 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a7da5-77b1-4dfe-9461-a0945559ea9c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:08.770852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a7da5-77b1-4dfe-9461-a0945559ea9c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:08.789950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 552/1110] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:47:08 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:47:10.980486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85196e9a-194e-47e8-a953-8e74da5c910b None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:10.980924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85196e9a-194e-47e8-a953-8e74da5c910b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:10.981035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85196e9a-194e-47e8-a953-8e74da5c910b None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:10.982398 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 559/1111] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:10.988428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff72ed-a1e3-4e12-8c6c-20d80b8af8a0 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:10.988842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff72ed-a1e3-4e12-8c6c-20d80b8af8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:10.989205 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff72ed-a1e3-4e12-8c6c-20d80b8af8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:11.005571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-81ff72ed-a1e3-4e12-8c6c-20d80b8af8a0 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:11.006778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-81ff72ed-a1e3-4e12-8c6c-20d80b8af8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:11.049516 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:11.074254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 553/1112] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:47:11.097965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85196e9a-194e-47e8-a953-8e74da5c910b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:11.200637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85196e9a-194e-47e8-a953-8e74da5c910b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:11.213752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:11.235195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21955b92-7b98-4740-b2e1-b0f3895cd954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:11.236043 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21955b92-7b98-4740-b2e1-b0f3895cd954 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:11.236324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21955b92-7b98-4740-b2e1-b0f3895cd954 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:11.236597 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21955b92-7b98-4740-b2e1-b0f3895cd954 None admin] PATH_INFO: `/v3/roles/662cb9b5dff94cdb83f292ad20d1d430` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:11.238074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21955b92-7b98-4740-b2e1-b0f3895cd954 None admin] RBAC: Authorizing `identity:get_role(role_id=662cb9b5dff94cdb83f292ad20d1d430)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:11.239535 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21955b92-7b98-4740-b2e1-b0f3895cd954 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:11.240884 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 560/1113] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:47:11 2022] GET /identity/v3/roles/662cb9b5dff94cdb83f292ad20d1d430 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:11.246721 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff72ed-a1e3-4e12-8c6c-20d80b8af8a0 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:11.348664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff72ed-a1e3-4e12-8c6c-20d80b8af8a0 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:11.381978 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b2a7c-ae33-474a-a346-b858e77d54fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:11.382920 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b2a7c-ae33-474a-a346-b858e77d54fc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:11.383220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b2a7c-ae33-474a-a346-b858e77d54fc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:11.383504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b2a7c-ae33-474a-a346-b858e77d54fc None admin] PATH_INFO: `/v3/users/9eb77e1c2640474486994a7a5c22f2a8` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:11.397563 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b2a7c-ae33-474a-a346-b858e77d54fc None admin] RBAC: Authorizing `identity:get_user(user_id=9eb77e1c2640474486994a7a5c22f2a8)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:11.399109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b2a7c-ae33-474a-a346-b858e77d54fc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:11.400811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 554/1114] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:47:11 2022] GET /identity/v3/users/9eb77e1c2640474486994a7a5c22f2a8 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:11.406600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21955b92-7b98-4740-b2e1-b0f3895cd954 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:11.506347 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21955b92-7b98-4740-b2e1-b0f3895cd954 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:11.540362 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bc14c-d8e2-4319-98f8-9500a87e8e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:11.541020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bc14c-d8e2-4319-98f8-9500a87e8e21 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:11.541327 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bc14c-d8e2-4319-98f8-9500a87e8e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:11.541624 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bc14c-d8e2-4319-98f8-9500a87e8e21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:11.590738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:11.610692 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bc14c-d8e2-4319-98f8-9500a87e8e21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:11.612161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bc14c-d8e2-4319-98f8-9500a87e8e21 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:11.710166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 561/1115] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:47:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:47:11.716201 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b2a7c-ae33-474a-a346-b858e77d54fc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:11.749991 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:11.813786 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b2a7c-ae33-474a-a346-b858e77d54fc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:11.847243 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b92bcb-0f1f-4421-8663-21398c374d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:11.848152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b92bcb-0f1f-4421-8663-21398c374d4e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:11.848439 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b92bcb-0f1f-4421-8663-21398c374d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:11.848718 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b92bcb-0f1f-4421-8663-21398c374d4e None admin] PATH_INFO: `/v3/projects/7a09ae933f9e40c1ab95266084ddff97` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:11.849895 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b92bcb-0f1f-4421-8663-21398c374d4e None admin] RBAC: Authorizing `identity:get_project(project_id=7a09ae933f9e40c1ab95266084ddff97)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:11.851349 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b92bcb-0f1f-4421-8663-21398c374d4e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:11.853042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 555/1116] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:47:11 2022] GET /identity/v3/projects/7a09ae933f9e40c1ab95266084ddff97 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:11.859557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bc14c-d8e2-4319-98f8-9500a87e8e21 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:11.938780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:11.959153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bc14c-d8e2-4319-98f8-9500a87e8e21 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:11.992620 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eff1a1-f977-42ad-a76f-5bdb9905c640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:11.993267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eff1a1-f977-42ad-a76f-5bdb9905c640 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:11.993550 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eff1a1-f977-42ad-a76f-5bdb9905c640 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:11.993826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eff1a1-f977-42ad-a76f-5bdb9905c640 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:11.994674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eff1a1-f977-42ad-a76f-5bdb9905c640 None admin] RBAC: Adding query filter params (role.id=662cb9b5dff94cdb83f292ad20d1d430, scope.project.id=7a09ae933f9e40c1ab95266084ddff97, user.id=9eb77e1c2640474486994a7a5c22f2a8) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:11.994988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eff1a1-f977-42ad-a76f-5bdb9905c640 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:11.996404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eff1a1-f977-42ad-a76f-5bdb9905c640 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:12.001796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 562/1117] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:47:11 2022] GET /identity/v3/role_assignments?user.id=9eb77e1c2640474486994a7a5c22f2a8&scope.project.id=7a09ae933f9e40c1ab95266084ddff97&role.id=662cb9b5dff94cdb83f292ad20d1d430 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:14.132119 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f51ade-7d20-4215-8abd-502ae8aa27f4 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:14.132652 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f51ade-7d20-4215-8abd-502ae8aa27f4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:14.132781 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f51ade-7d20-4215-8abd-502ae8aa27f4 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:14.134122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 556/1118] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:14.140002 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a8b8f-4a91-4b85-ba5e-c8f6f75ec852 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:14.140424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a8b8f-4a91-4b85-ba5e-c8f6f75ec852 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:14.140555 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a8b8f-4a91-4b85-ba5e-c8f6f75ec852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:14.158330 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-612a8b8f-4a91-4b85-ba5e-c8f6f75ec852 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:14.160094 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-612a8b8f-4a91-4b85-ba5e-c8f6f75ec852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:14.225776 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 563/1119] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:47:14.248217 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f51ade-7d20-4215-8abd-502ae8aa27f4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:14.348560 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f51ade-7d20-4215-8abd-502ae8aa27f4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:14.351561 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:14.382061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac0662-01f3-40ff-bea2-3b2d8d9dda38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:14.382927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac0662-01f3-40ff-bea2-3b2d8d9dda38 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:14.383210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac0662-01f3-40ff-bea2-3b2d8d9dda38 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:14.383479 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac0662-01f3-40ff-bea2-3b2d8d9dda38 None admin] PATH_INFO: `/v3/roles/662cb9b5dff94cdb83f292ad20d1d430` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:14.384933 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac0662-01f3-40ff-bea2-3b2d8d9dda38 None admin] RBAC: Authorizing `identity:get_role(role_id=662cb9b5dff94cdb83f292ad20d1d430)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:14.386278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac0662-01f3-40ff-bea2-3b2d8d9dda38 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:14.387662 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 557/1120] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:47:14 2022] GET /identity/v3/roles/662cb9b5dff94cdb83f292ad20d1d430 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:14.394220 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a8b8f-4a91-4b85-ba5e-c8f6f75ec852 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:14.467889 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:14.492781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a8b8f-4a91-4b85-ba5e-c8f6f75ec852 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:14.526150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad9195-7ce4-4321-9fb0-a007f54b3ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:14.527038 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad9195-7ce4-4321-9fb0-a007f54b3ecd None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:14.527326 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad9195-7ce4-4321-9fb0-a007f54b3ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:14.528546 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad9195-7ce4-4321-9fb0-a007f54b3ecd None admin] PATH_INFO: `/v3/users/9eb77e1c2640474486994a7a5c22f2a8` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:14.529738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad9195-7ce4-4321-9fb0-a007f54b3ecd None admin] RBAC: Authorizing `identity:get_user(user_id=9eb77e1c2640474486994a7a5c22f2a8)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:14.531201 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad9195-7ce4-4321-9fb0-a007f54b3ecd None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:14.532830 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 564/1121] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:47:14 2022] GET /identity/v3/users/9eb77e1c2640474486994a7a5c22f2a8 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:14.538856 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac0662-01f3-40ff-bea2-3b2d8d9dda38 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:14.638154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac0662-01f3-40ff-bea2-3b2d8d9dda38 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:14.671860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05298e54-6194-4b8d-b0a7-0ed661712b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:14.672705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05298e54-6194-4b8d-b0a7-0ed661712b02 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:14.672985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05298e54-6194-4b8d-b0a7-0ed661712b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:14.673259 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05298e54-6194-4b8d-b0a7-0ed661712b02 None admin] PATH_INFO: `/v3/projects/7a09ae933f9e40c1ab95266084ddff97` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:14.674416 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05298e54-6194-4b8d-b0a7-0ed661712b02 None admin] RBAC: Authorizing `identity:get_project(project_id=7a09ae933f9e40c1ab95266084ddff97)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:14.675852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05298e54-6194-4b8d-b0a7-0ed661712b02 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:14.677647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 558/1122] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:47:14 2022] GET /identity/v3/projects/7a09ae933f9e40c1ab95266084ddff97 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:14.683490 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad9195-7ce4-4321-9fb0-a007f54b3ecd None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:14.781460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad9195-7ce4-4321-9fb0-a007f54b3ecd None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:14.813438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b776a-5b69-4b0f-8d52-c25b7728f379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:14.814302 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b776a-5b69-4b0f-8d52-c25b7728f379 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:14.814599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b776a-5b69-4b0f-8d52-c25b7728f379 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:14.814881 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b776a-5b69-4b0f-8d52-c25b7728f379 None admin] PATH_INFO: `/v3/projects/7a09ae933f9e40c1ab95266084ddff97/users/9eb77e1c2640474486994a7a5c22f2a8/roles/662cb9b5dff94cdb83f292ad20d1d430` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:14.817680 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b776a-5b69-4b0f-8d52-c25b7728f379 None admin] RBAC: Authorizing `identity:create_grant(project_id=7a09ae933f9e40c1ab95266084ddff97, user_id=9eb77e1c2640474486994a7a5c22f2a8, role_id=662cb9b5dff94cdb83f292ad20d1d430)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:14.819129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b776a-5b69-4b0f-8d52-c25b7728f379 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:14.835509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 565/1123] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:47:14 2022] PUT /identity/v3/projects/7a09ae933f9e40c1ab95266084ddff97/users/9eb77e1c2640474486994a7a5c22f2a8/roles/662cb9b5dff94cdb83f292ad20d1d430 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:47:16.970409 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1177d5c-2da4-44e3-83a8-3d9567694335 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:16.970860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1177d5c-2da4-44e3-83a8-3d9567694335 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:16.970980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1177d5c-2da4-44e3-83a8-3d9567694335 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:16.973130 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 559/1124] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:16.978365 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c717a-de10-46a6-b994-759593281f27 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:16.978803 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c717a-de10-46a6-b994-759593281f27 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:16.978969 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c717a-de10-46a6-b994-759593281f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:16.983969 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:16.997587 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f53c717a-de10-46a6-b994-759593281f27 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:16.998807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f53c717a-de10-46a6-b994-759593281f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:17.065222 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 566/1125] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:47:17.089496 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1177d5c-2da4-44e3-83a8-3d9567694335 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:17.134412 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:17.193593 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1177d5c-2da4-44e3-83a8-3d9567694335 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:17.227435 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668432b8-9395-49f8-87d0-b91d53c1ba4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:17.228203 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668432b8-9395-49f8-87d0-b91d53c1ba4b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:17.228484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668432b8-9395-49f8-87d0-b91d53c1ba4b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:17.228753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668432b8-9395-49f8-87d0-b91d53c1ba4b None admin] PATH_INFO: `/v3/roles/662cb9b5dff94cdb83f292ad20d1d430` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:17.230198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668432b8-9395-49f8-87d0-b91d53c1ba4b None admin] RBAC: Authorizing `identity:get_role(role_id=662cb9b5dff94cdb83f292ad20d1d430)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:17.231583 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668432b8-9395-49f8-87d0-b91d53c1ba4b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:17.232949 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 560/1126] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:47:17 2022] GET /identity/v3/roles/662cb9b5dff94cdb83f292ad20d1d430 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:17.239103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c717a-de10-46a6-b994-759593281f27 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:17.337442 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c717a-de10-46a6-b994-759593281f27 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:17.367063 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:17.372326 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfe695-c73e-4e98-80b2-9e6190ddbd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:17.373182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfe695-c73e-4e98-80b2-9e6190ddbd0e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:17.373458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfe695-c73e-4e98-80b2-9e6190ddbd0e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:17.373727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfe695-c73e-4e98-80b2-9e6190ddbd0e None admin] PATH_INFO: `/v3/users/9eb77e1c2640474486994a7a5c22f2a8` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:17.374920 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfe695-c73e-4e98-80b2-9e6190ddbd0e None admin] RBAC: Authorizing `identity:get_user(user_id=9eb77e1c2640474486994a7a5c22f2a8)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:17.376378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfe695-c73e-4e98-80b2-9e6190ddbd0e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:17.378200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 567/1127] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:47:17 2022] GET /identity/v3/users/9eb77e1c2640474486994a7a5c22f2a8 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:17.383993 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668432b8-9395-49f8-87d0-b91d53c1ba4b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:17.483441 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668432b8-9395-49f8-87d0-b91d53c1ba4b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:17.492627 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:17.516743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99da340-ba0a-4f59-bd0e-4c9b08cad0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:17.517384 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99da340-ba0a-4f59-bd0e-4c9b08cad0ea None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:17.517664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99da340-ba0a-4f59-bd0e-4c9b08cad0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:17.517939 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99da340-ba0a-4f59-bd0e-4c9b08cad0ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:17.583544 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99da340-ba0a-4f59-bd0e-4c9b08cad0ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:17.584965 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99da340-ba0a-4f59-bd0e-4c9b08cad0ea None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:17.678246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 561/1128] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:47:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:47:17.684631 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfe695-c73e-4e98-80b2-9e6190ddbd0e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:17.783707 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfe695-c73e-4e98-80b2-9e6190ddbd0e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:17.816079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0962cfd-88e6-4331-965b-18cd9579e203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:17.817082 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0962cfd-88e6-4331-965b-18cd9579e203 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:17.817360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0962cfd-88e6-4331-965b-18cd9579e203 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:17.817629 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0962cfd-88e6-4331-965b-18cd9579e203 None admin] PATH_INFO: `/v3/projects/7a09ae933f9e40c1ab95266084ddff97` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:17.818838 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0962cfd-88e6-4331-965b-18cd9579e203 None admin] RBAC: Authorizing `identity:get_project(project_id=7a09ae933f9e40c1ab95266084ddff97)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:17.820262 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0962cfd-88e6-4331-965b-18cd9579e203 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:17.821930 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 568/1129] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:47:17 2022] GET /identity/v3/projects/7a09ae933f9e40c1ab95266084ddff97 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:17.828165 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99da340-ba0a-4f59-bd0e-4c9b08cad0ea None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:17.839111 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:17.927856 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99da340-ba0a-4f59-bd0e-4c9b08cad0ea None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:17.962305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e2c03-be13-4811-9ea6-7a417685a990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:17.962993 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e2c03-be13-4811-9ea6-7a417685a990 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:17.963281 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e2c03-be13-4811-9ea6-7a417685a990 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:17.963558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e2c03-be13-4811-9ea6-7a417685a990 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:17.964383 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e2c03-be13-4811-9ea6-7a417685a990 None admin] RBAC: Adding query filter params (role.id=662cb9b5dff94cdb83f292ad20d1d430, scope.project.id=7a09ae933f9e40c1ab95266084ddff97, user.id=9eb77e1c2640474486994a7a5c22f2a8) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:17.964696 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e2c03-be13-4811-9ea6-7a417685a990 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:17.966113 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e2c03-be13-4811-9ea6-7a417685a990 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:17.972816 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 562/1130] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:47:17 2022] GET /identity/v3/role_assignments?user.id=9eb77e1c2640474486994a7a5c22f2a8&scope.project.id=7a09ae933f9e40c1ab95266084ddff97&role.id=662cb9b5dff94cdb83f292ad20d1d430 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:20.112638 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7745556-b4ee-405b-b87c-1fa7c6413612 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:20.113226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7745556-b4ee-405b-b87c-1fa7c6413612 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:20.113226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7745556-b4ee-405b-b87c-1fa7c6413612 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:20.114558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 569/1131] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:20.119856 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f90a18-7f04-4340-a528-32d4ca87d384 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:20.120281 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f90a18-7f04-4340-a528-32d4ca87d384 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:20.120420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f90a18-7f04-4340-a528-32d4ca87d384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:20.137594 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-78f90a18-7f04-4340-a528-32d4ca87d384 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:20.138796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-78f90a18-7f04-4340-a528-32d4ca87d384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:20.208686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 563/1132] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:47:20.235632 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7745556-b4ee-405b-b87c-1fa7c6413612 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:20.303983 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:20.338914 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7745556-b4ee-405b-b87c-1fa7c6413612 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:20.372264 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb2944-a3e1-476c-b358-34a37b1a2f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:20.373043 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb2944-a3e1-476c-b358-34a37b1a2f3d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:20.373328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb2944-a3e1-476c-b358-34a37b1a2f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:20.373606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb2944-a3e1-476c-b358-34a37b1a2f3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:20.374796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb2944-a3e1-476c-b358-34a37b1a2f3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:20.376219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb2944-a3e1-476c-b358-34a37b1a2f3d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:20.378464 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 570/1133] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:47:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:20.384933 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f90a18-7f04-4340-a528-32d4ca87d384 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:20.387122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:20.481876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f90a18-7f04-4340-a528-32d4ca87d384 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:20.515548 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e3075-b320-4e19-a916-f3a2b46e2f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:20.516190 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e3075-b320-4e19-a916-f3a2b46e2f8f None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:20.516738 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e3075-b320-4e19-a916-f3a2b46e2f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:20.517192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e3075-b320-4e19-a916-f3a2b46e2f8f None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:20.517903 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e3075-b320-4e19-a916-f3a2b46e2f8f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:20.519439 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e3075-b320-4e19-a916-f3a2b46e2f8f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:20.548979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 564/1134] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Jun 16 18:47:20 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:47:22.736068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddf702-a355-4ee2-8a54-42d45086c004 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:22.736628 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddf702-a355-4ee2-8a54-42d45086c004 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:22.736628 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddf702-a355-4ee2-8a54-42d45086c004 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:22.738612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 571/1135] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:22.743823 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44016b9-0c55-4abc-9e12-39b848357efb None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:22.744262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44016b9-0c55-4abc-9e12-39b848357efb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:22.744380 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44016b9-0c55-4abc-9e12-39b848357efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:22.761378 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f44016b9-0c55-4abc-9e12-39b848357efb None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:22.762533 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f44016b9-0c55-4abc-9e12-39b848357efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:22.811169 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:22.829078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 565/1136] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:47:22.854254 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ddf702-a355-4ee2-8a54-42d45086c004 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:22.962556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ddf702-a355-4ee2-8a54-42d45086c004 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:22.998931 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9d156-1473-4e66-9e29-01815dd3937b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:22.999651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9d156-1473-4e66-9e29-01815dd3937b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:22.999938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9d156-1473-4e66-9e29-01815dd3937b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:23.000219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9d156-1473-4e66-9e29-01815dd3937b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:23.001406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9d156-1473-4e66-9e29-01815dd3937b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:23.003430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9d156-1473-4e66-9e29-01815dd3937b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:23.005065 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 572/1137] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:47:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:23.011040 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44016b9-0c55-4abc-9e12-39b848357efb None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:23.110254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44016b9-0c55-4abc-9e12-39b848357efb None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:23.146049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1420ee3-5ce5-4f0e-9a15-31ec57069183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:23.146758 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1420ee3-5ce5-4f0e-9a15-31ec57069183 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:23.147044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1420ee3-5ce5-4f0e-9a15-31ec57069183 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:23.147317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1420ee3-5ce5-4f0e-9a15-31ec57069183 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:23.148020 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1420ee3-5ce5-4f0e-9a15-31ec57069183 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:23.149450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1420ee3-5ce5-4f0e-9a15-31ec57069183 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:23.170682 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 566/1138] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:47:23 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:47:25.364177 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17001c1-a4dd-49fb-83bf-2eefb929abae None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:25.364705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17001c1-a4dd-49fb-83bf-2eefb929abae None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:25.364705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17001c1-a4dd-49fb-83bf-2eefb929abae None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:25.366062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 573/1139] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:25.371318 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3ea2e-5039-4045-9d9f-eb5e4634a56a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:25.371713 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3ea2e-5039-4045-9d9f-eb5e4634a56a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:25.371869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3ea2e-5039-4045-9d9f-eb5e4634a56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:25.389029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-17b3ea2e-5039-4045-9d9f-eb5e4634a56a None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:25.390221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-17b3ea2e-5039-4045-9d9f-eb5e4634a56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:25.424060 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:25.458291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 567/1140] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:47:25.481642 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17001c1-a4dd-49fb-83bf-2eefb929abae None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:25.485432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:25.586301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17001c1-a4dd-49fb-83bf-2eefb929abae None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:25.619371 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64de4a5-b714-436f-837f-cf67c8469ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:25.620084 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64de4a5-b714-436f-837f-cf67c8469ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:25.620363 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64de4a5-b714-436f-837f-cf67c8469ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:25.620630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64de4a5-b714-436f-837f-cf67c8469ca3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:25.631124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64de4a5-b714-436f-837f-cf67c8469ca3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:47:25.638639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-d64de4a5-b714-436f-837f-cf67c8469ca3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:47:25.638639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:25.638639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:25.638639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:25.638639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:25.638639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:25.638639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:25.638639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:25.638639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:25.638639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:25.638639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:25.638639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:25.638639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:47:25.638639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:47:25.638639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:47:25.638639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:47:25.639746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:47:25.639746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:47:25.639746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:47:25.639746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:47:25.639746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:25.639746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:25.639746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:25.639746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:25.639746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:25.639746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:25.639746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:25.639746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:25.639746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:25.639746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:25.639746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:25.639746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:25.640639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:25.640639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:25.640639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:25.640639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:25.640639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:47:25.640639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:47:25.640639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:47:25.640639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:47:25.640639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:47:25.640639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:47:25.640639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:47:25.640639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:47:25.640639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 574/1141] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:47:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:47:25.646838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3ea2e-5039-4045-9d9f-eb5e4634a56a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:25.745585 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3ea2e-5039-4045-9d9f-eb5e4634a56a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:25.780430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf13a1-722e-4c5e-8fa6-126cfe66d009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:25.781091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf13a1-722e-4c5e-8fa6-126cfe66d009 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:25.781375 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf13a1-722e-4c5e-8fa6-126cfe66d009 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:25.781651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf13a1-722e-4c5e-8fa6-126cfe66d009 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:25.782393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf13a1-722e-4c5e-8fa6-126cfe66d009 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:25.782734 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf13a1-722e-4c5e-8fa6-126cfe66d009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:25.785165 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf13a1-722e-4c5e-8fa6-126cfe66d009 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:25.793109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 568/1142] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:47:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:25.799001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64de4a5-b714-436f-837f-cf67c8469ca3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:25.861445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:25.899976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64de4a5-b714-436f-837f-cf67c8469ca3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:25.932242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0fd6b-45d7-4881-b856-d05c99d90a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:25.933099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0fd6b-45d7-4881-b856-d05c99d90a9c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:25.933383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0fd6b-45d7-4881-b856-d05c99d90a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:25.933660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0fd6b-45d7-4881-b856-d05c99d90a9c None admin] PATH_INFO: `/v3/users/00541fffac1045df9c0fe03fdc5c6de1` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:25.947166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0fd6b-45d7-4881-b856-d05c99d90a9c None admin] RBAC: Authorizing `identity:get_user(user_id=00541fffac1045df9c0fe03fdc5c6de1)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:25.948599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0fd6b-45d7-4881-b856-d05c99d90a9c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:25.950266 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 575/1143] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:47:25 2022] GET /identity/v3/users/00541fffac1045df9c0fe03fdc5c6de1 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:25.956560 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf13a1-722e-4c5e-8fa6-126cfe66d009 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:25.987316 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:26.054671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf13a1-722e-4c5e-8fa6-126cfe66d009 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:26.086693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5f50c-192e-46eb-9823-e7bd6b81ba3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:26.087326 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5f50c-192e-46eb-9823-e7bd6b81ba3a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:26.087625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5f50c-192e-46eb-9823-e7bd6b81ba3a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:26.087876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5f50c-192e-46eb-9823-e7bd6b81ba3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:26.154091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d5f50c-192e-46eb-9823-e7bd6b81ba3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:26.155501 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d5f50c-192e-46eb-9823-e7bd6b81ba3a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:26.184632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:26.254127 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 569/1144] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:47:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:47:26.260102 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0fd6b-45d7-4881-b856-d05c99d90a9c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:26.356984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0fd6b-45d7-4881-b856-d05c99d90a9c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:26.381358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:26.393432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4818feb-9d58-436a-ae41-d799f86ef1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:26.394288 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4818feb-9d58-436a-ae41-d799f86ef1db None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:26.394597 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4818feb-9d58-436a-ae41-d799f86ef1db None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:26.394873 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4818feb-9d58-436a-ae41-d799f86ef1db None admin] PATH_INFO: `/v3/projects/dde328b1a72649fe940b1eb5c1db6e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:26.396059 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4818feb-9d58-436a-ae41-d799f86ef1db None admin] RBAC: Authorizing `identity:get_project(project_id=dde328b1a72649fe940b1eb5c1db6e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:26.397483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4818feb-9d58-436a-ae41-d799f86ef1db None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:26.399191 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 576/1145] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:47:26 2022] GET /identity/v3/projects/dde328b1a72649fe940b1eb5c1db6e62 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:26.405293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5f50c-192e-46eb-9823-e7bd6b81ba3a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:26.503198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5f50c-192e-46eb-9823-e7bd6b81ba3a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:26.531582 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:26.537534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36809304-d35e-4ec8-b729-a90433196031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:26.538182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36809304-d35e-4ec8-b729-a90433196031 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:26.538466 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36809304-d35e-4ec8-b729-a90433196031 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:26.538778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36809304-d35e-4ec8-b729-a90433196031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:26.539603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36809304-d35e-4ec8-b729-a90433196031 None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=dde328b1a72649fe940b1eb5c1db6e62, user.id=00541fffac1045df9c0fe03fdc5c6de1) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:26.539915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36809304-d35e-4ec8-b729-a90433196031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:26.541328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36809304-d35e-4ec8-b729-a90433196031 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:26.546750 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 570/1146] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:47:26 2022] GET /identity/v3/role_assignments?user.id=00541fffac1045df9c0fe03fdc5c6de1&scope.project.id=dde328b1a72649fe940b1eb5c1db6e62&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:28.628341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f004ca-398f-4012-b0dd-9d9bf84e09b9 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:28.628877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f004ca-398f-4012-b0dd-9d9bf84e09b9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:28.628877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f004ca-398f-4012-b0dd-9d9bf84e09b9 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:28.630226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 577/1147] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:28.635334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14a542-5739-4749-8bb2-24253db1c1e2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:28.635730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14a542-5739-4749-8bb2-24253db1c1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:28.635891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14a542-5739-4749-8bb2-24253db1c1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:28.655024 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8d14a542-5739-4749-8bb2-24253db1c1e2 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:28.656193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8d14a542-5739-4749-8bb2-24253db1c1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:28.723326 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 571/1148] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:47:28.746008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f004ca-398f-4012-b0dd-9d9bf84e09b9 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:28.846937 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f004ca-398f-4012-b0dd-9d9bf84e09b9 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:28.880437 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a41304-6a04-4f05-9996-61816c7b3389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:28.881160 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a41304-6a04-4f05-9996-61816c7b3389 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:28.881443 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a41304-6a04-4f05-9996-61816c7b3389 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:28.881719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a41304-6a04-4f05-9996-61816c7b3389 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:28.892003 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a41304-6a04-4f05-9996-61816c7b3389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:47:28.899773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-07a41304-6a04-4f05-9996-61816c7b3389 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:47:28.899773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:28.899773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:28.899773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:28.899773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:28.899773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:28.899773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:28.899773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:28.899773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:28.899773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:28.899773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:28.899773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:28.899773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:47:28.899773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:47:28.899773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:47:28.899773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:47:28.900949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:47:28.900949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:47:28.900949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:47:28.900949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:47:28.900949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:28.900949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:28.900949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:28.900949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:28.900949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:28.900949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:28.900949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:28.900949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:28.900949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:28.900949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:28.900949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:28.900949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:28.901976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:28.901976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:28.901976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:28.901976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:28.901976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:47:28.901976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:47:28.901976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:47:28.901976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:47:28.901976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:47:28.901976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:47:28.901976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:47:28.901976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:47:28.901976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 578/1149] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:47:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:47:28.907073 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14a542-5739-4749-8bb2-24253db1c1e2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:29.005685 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14a542-5739-4749-8bb2-24253db1c1e2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:29.030559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:29.040181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a1114-3f9b-440c-bd0d-314058597b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:29.040838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a1114-3f9b-440c-bd0d-314058597b56 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:29.041118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a1114-3f9b-440c-bd0d-314058597b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:29.041389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a1114-3f9b-440c-bd0d-314058597b56 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:29.042125 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a1114-3f9b-440c-bd0d-314058597b56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:29.042428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a1114-3f9b-440c-bd0d-314058597b56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:29.043818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a1114-3f9b-440c-bd0d-314058597b56 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:29.052447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 572/1150] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:47:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:29.058235 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a41304-6a04-4f05-9996-61816c7b3389 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:29.106723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:29.155775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a41304-6a04-4f05-9996-61816c7b3389 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:29.188726 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd0ba4-d227-4f0c-a2a9-24b57ad46ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:29.189586 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd0ba4-d227-4f0c-a2a9-24b57ad46ffa None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:29.190099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd0ba4-d227-4f0c-a2a9-24b57ad46ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:29.190462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd0ba4-d227-4f0c-a2a9-24b57ad46ffa None admin] PATH_INFO: `/v3/users/00541fffac1045df9c0fe03fdc5c6de1` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:29.191714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd0ba4-d227-4f0c-a2a9-24b57ad46ffa None admin] RBAC: Authorizing `identity:get_user(user_id=00541fffac1045df9c0fe03fdc5c6de1)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:29.193145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd0ba4-d227-4f0c-a2a9-24b57ad46ffa None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:29.194830 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 579/1151] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:47:29 2022] GET /identity/v3/users/00541fffac1045df9c0fe03fdc5c6de1 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:29.200561 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a1114-3f9b-440c-bd0d-314058597b56 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:29.302035 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a1114-3f9b-440c-bd0d-314058597b56 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:29.335853 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a116c6a-3c17-40f8-aa10-555ad261582b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:29.336701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a116c6a-3c17-40f8-aa10-555ad261582b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:29.336985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a116c6a-3c17-40f8-aa10-555ad261582b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:29.337260 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a116c6a-3c17-40f8-aa10-555ad261582b None admin] PATH_INFO: `/v3/projects/dde328b1a72649fe940b1eb5c1db6e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:29.338426 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a116c6a-3c17-40f8-aa10-555ad261582b None admin] RBAC: Authorizing `identity:get_project(project_id=dde328b1a72649fe940b1eb5c1db6e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:29.340029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a116c6a-3c17-40f8-aa10-555ad261582b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:29.341726 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 573/1152] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:47:29 2022] GET /identity/v3/projects/dde328b1a72649fe940b1eb5c1db6e62 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:29.348931 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd0ba4-d227-4f0c-a2a9-24b57ad46ffa None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:29.452256 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd0ba4-d227-4f0c-a2a9-24b57ad46ffa None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:29.469217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:29.486586 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d002fe9-3106-4e59-a2cd-7f60117ca9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:29.487481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d002fe9-3106-4e59-a2cd-7f60117ca9eb None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:29.487760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d002fe9-3106-4e59-a2cd-7f60117ca9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:29.488033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d002fe9-3106-4e59-a2cd-7f60117ca9eb None admin] PATH_INFO: `/v3/projects/dde328b1a72649fe940b1eb5c1db6e62/users/00541fffac1045df9c0fe03fdc5c6de1/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:29.489980 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d002fe9-3106-4e59-a2cd-7f60117ca9eb None admin] RBAC: Authorizing `identity:create_grant(project_id=dde328b1a72649fe940b1eb5c1db6e62, user_id=00541fffac1045df9c0fe03fdc5c6de1, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:29.491438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d002fe9-3106-4e59-a2cd-7f60117ca9eb None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:29.504096 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 580/1153] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:47:29 2022] PUT /identity/v3/projects/dde328b1a72649fe940b1eb5c1db6e62/users/00541fffac1045df9c0fe03fdc5c6de1/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:47:31.751370 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889bfdf-0c82-420f-a1d9-de8fdfcd590e None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:31.751921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889bfdf-0c82-420f-a1d9-de8fdfcd590e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:31.751921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889bfdf-0c82-420f-a1d9-de8fdfcd590e None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:31.753437 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 574/1154] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:31.758825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444d5be-eacc-44e5-b7e4-366c276d3756 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:31.759246 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444d5be-eacc-44e5-b7e4-366c276d3756 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:31.759440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444d5be-eacc-44e5-b7e4-366c276d3756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:31.776930 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f444d5be-eacc-44e5-b7e4-366c276d3756 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:31.778101 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f444d5be-eacc-44e5-b7e4-366c276d3756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:31.843730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 581/1155] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:47:31.866749 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889bfdf-0c82-420f-a1d9-de8fdfcd590e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:31.923517 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:31.967733 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889bfdf-0c82-420f-a1d9-de8fdfcd590e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:32.000720 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c73b24-fe44-43f3-8a7e-3a69fdf579bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:32.001546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c73b24-fe44-43f3-8a7e-3a69fdf579bd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:32.001775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c73b24-fe44-43f3-8a7e-3a69fdf579bd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:32.002068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c73b24-fe44-43f3-8a7e-3a69fdf579bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:32.012258 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c73b24-fe44-43f3-8a7e-3a69fdf579bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:47:32.020246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e0c73b24-fe44-43f3-8a7e-3a69fdf579bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:47:32.020246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:32.020246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:32.020246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:32.020246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:32.020246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:32.020246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:32.020246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:32.020246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:32.020246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:32.020246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:32.020246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:32.020246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:47:32.020246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:47:32.020246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:47:32.020246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:47:32.021345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:47:32.021345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:47:32.021345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:47:32.021345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:47:32.021345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:32.021345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:32.021345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:32.021345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:32.021345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:32.021345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:32.021345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:32.021345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:32.021345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:32.021345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:32.021345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:32.021345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:32.022660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:32.022660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:32.022660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:32.022660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:32.022660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:47:32.022660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:47:32.022660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:47:32.022660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:47:32.022660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:47:32.022660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:47:32.022660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:47:32.022660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:47:32.022660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 575/1156] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:47:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:47:32.027531 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444d5be-eacc-44e5-b7e4-366c276d3756 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:32.125438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444d5be-eacc-44e5-b7e4-366c276d3756 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:32.134350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:32.161336 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4cf03c-b8f2-4068-b6f0-51ac05b371c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:32.161992 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cf03c-b8f2-4068-b6f0-51ac05b371c7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:32.162277 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cf03c-b8f2-4068-b6f0-51ac05b371c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:32.162596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cf03c-b8f2-4068-b6f0-51ac05b371c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:32.163363 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4cf03c-b8f2-4068-b6f0-51ac05b371c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:32.163694 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4cf03c-b8f2-4068-b6f0-51ac05b371c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:32.165108 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4cf03c-b8f2-4068-b6f0-51ac05b371c7 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:32.173049 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 582/1157] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:47:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:32.179434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c73b24-fe44-43f3-8a7e-3a69fdf579bd None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:32.276151 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c73b24-fe44-43f3-8a7e-3a69fdf579bd None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:32.292506 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:32.309961 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52fe54-4fb5-40b3-ac24-99f4e76fbbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:32.310842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52fe54-4fb5-40b3-ac24-99f4e76fbbde None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:32.311129 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52fe54-4fb5-40b3-ac24-99f4e76fbbde None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:32.311404 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52fe54-4fb5-40b3-ac24-99f4e76fbbde None admin] PATH_INFO: `/v3/users/00541fffac1045df9c0fe03fdc5c6de1` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:32.312575 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b52fe54-4fb5-40b3-ac24-99f4e76fbbde None admin] RBAC: Authorizing `identity:get_user(user_id=00541fffac1045df9c0fe03fdc5c6de1)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:32.313985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b52fe54-4fb5-40b3-ac24-99f4e76fbbde None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:32.315820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 576/1158] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:47:32 2022] GET /identity/v3/users/00541fffac1045df9c0fe03fdc5c6de1 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:32.321612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4cf03c-b8f2-4068-b6f0-51ac05b371c7 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:32.426458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4cf03c-b8f2-4068-b6f0-51ac05b371c7 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:32.460786 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c1694-c035-4d2e-9d37-b8b4cf0a05d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:32.461473 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c1694-c035-4d2e-9d37-b8b4cf0a05d5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:32.461754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c1694-c035-4d2e-9d37-b8b4cf0a05d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:32.462035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c1694-c035-4d2e-9d37-b8b4cf0a05d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:32.501168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:32.530308 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c1694-c035-4d2e-9d37-b8b4cf0a05d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:32.531723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c1694-c035-4d2e-9d37-b8b4cf0a05d5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:32.628371 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 583/1159] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:47:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:47:32.634213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52fe54-4fb5-40b3-ac24-99f4e76fbbde None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:32.740342 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52fe54-4fb5-40b3-ac24-99f4e76fbbde None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:32.775019 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b385667-7d7f-49ad-8b24-77e6aefcdbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:32.775867 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b385667-7d7f-49ad-8b24-77e6aefcdbef None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:32.776148 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b385667-7d7f-49ad-8b24-77e6aefcdbef None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:32.776424 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b385667-7d7f-49ad-8b24-77e6aefcdbef None admin] PATH_INFO: `/v3/projects/dde328b1a72649fe940b1eb5c1db6e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:32.777593 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b385667-7d7f-49ad-8b24-77e6aefcdbef None admin] RBAC: Authorizing `identity:get_project(project_id=dde328b1a72649fe940b1eb5c1db6e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:32.779037 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b385667-7d7f-49ad-8b24-77e6aefcdbef None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:32.780759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 577/1160] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:47:32 2022] GET /identity/v3/projects/dde328b1a72649fe940b1eb5c1db6e62 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:32.787736 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c1694-c035-4d2e-9d37-b8b4cf0a05d5 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:32.855202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:32.885770 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c1694-c035-4d2e-9d37-b8b4cf0a05d5 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:32.919483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff1a97-07e1-4828-9971-9af26e334871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:32.920122 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff1a97-07e1-4828-9971-9af26e334871 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:32.920403 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff1a97-07e1-4828-9971-9af26e334871 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:32.920680 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff1a97-07e1-4828-9971-9af26e334871 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:32.921494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff1a97-07e1-4828-9971-9af26e334871 None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=dde328b1a72649fe940b1eb5c1db6e62, user.id=00541fffac1045df9c0fe03fdc5c6de1) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:32.921807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff1a97-07e1-4828-9971-9af26e334871 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:32.923255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff1a97-07e1-4828-9971-9af26e334871 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:32.930111 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 584/1161] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:47:32 2022] GET /identity/v3/role_assignments?user.id=00541fffac1045df9c0fe03fdc5c6de1&scope.project.id=dde328b1a72649fe940b1eb5c1db6e62&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:35.079530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521d724-ada7-4790-b36a-1f1754a77360 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:35.080115 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521d724-ada7-4790-b36a-1f1754a77360 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:35.080293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521d724-ada7-4790-b36a-1f1754a77360 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:35.081597 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 578/1162] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:35.086988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0e4a3-9812-4eed-bb53-9aad33dc523c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:35.087389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0e4a3-9812-4eed-bb53-9aad33dc523c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:35.087578 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0e4a3-9812-4eed-bb53-9aad33dc523c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:35.104491 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-27d0e4a3-9812-4eed-bb53-9aad33dc523c None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:35.105726 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-27d0e4a3-9812-4eed-bb53-9aad33dc523c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:35.171861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 585/1163] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:47:35.194511 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521d724-ada7-4790-b36a-1f1754a77360 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:35.248245 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:35.295334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521d724-ada7-4790-b36a-1f1754a77360 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:35.327699 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566d868-0ef1-4692-b5aa-8579b68f4128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:35.328328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566d868-0ef1-4692-b5aa-8579b68f4128 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:35.328611 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566d868-0ef1-4692-b5aa-8579b68f4128 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:35.328883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566d868-0ef1-4692-b5aa-8579b68f4128 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:35.394986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d566d868-0ef1-4692-b5aa-8579b68f4128 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:35.396442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d566d868-0ef1-4692-b5aa-8579b68f4128 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:35.445083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:35.493535 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 579/1164] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:47:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:47:35.499579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0e4a3-9812-4eed-bb53-9aad33dc523c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:35.560557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:35.598436 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0e4a3-9812-4eed-bb53-9aad33dc523c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:35.632518 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69a73e-eae6-4d87-83ff-2fe867cc7c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:35.633309 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69a73e-eae6-4d87-83ff-2fe867cc7c28 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:35.633593 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69a73e-eae6-4d87-83ff-2fe867cc7c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:35.633871 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69a73e-eae6-4d87-83ff-2fe867cc7c28 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:35.640528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69a73e-eae6-4d87-83ff-2fe867cc7c28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:35.642264 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69a73e-eae6-4d87-83ff-2fe867cc7c28 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:35.659564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-bc69a73e-eae6-4d87-83ff-2fe867cc7c28 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 18:47:35.659564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:35.659564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 16 18:47:35.659564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 16 18:47:35.659564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:47:35.659564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:47:35.659564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:47:35.659564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:47:35.659564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 16 18:47:35.659564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:47:35.659564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:47:35.659564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:47:35.659564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:35.659564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:35.659564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:35.659564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:35.659564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:35.660854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:35.660854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:35.660854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:35.660854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:35.660854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:35.660854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:35.660854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:47:35.660854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:47:35.660854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:47:35.660854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:47:35.660854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:35.660854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:35.660854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:35.660854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:35.660854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:35.660854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:35.662190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:35.662190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:35.662190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:35.662190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:35.662190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:35.662190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:35.662190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:35.662190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:35.662190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:35.662190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:35.662190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 16 18:47:35.662190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:47:35.662190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 18:47:35.662190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:47:35.662190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 586/1165] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Jun 16 18:47:35 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:47:35.667053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566d868-0ef1-4692-b5aa-8579b68f4128 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:35.766285 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566d868-0ef1-4692-b5aa-8579b68f4128 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:35.804529 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647da846-7d1a-4935-95dd-0941e0b94503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:35.805193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647da846-7d1a-4935-95dd-0941e0b94503 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:35.805475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647da846-7d1a-4935-95dd-0941e0b94503 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:35.805750 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647da846-7d1a-4935-95dd-0941e0b94503 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:35.806468 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647da846-7d1a-4935-95dd-0941e0b94503 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:35.806810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647da846-7d1a-4935-95dd-0941e0b94503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:35.808167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647da846-7d1a-4935-95dd-0941e0b94503 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:35.815972 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 580/1166] 10.222.0.70 () {56 vars in 1115 bytes} [Thu Jun 16 18:47:35 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:37.938869 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3354bd-81cf-480b-8b11-ff3c510b6501 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:37.939465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3354bd-81cf-480b-8b11-ff3c510b6501 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:37.939465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3354bd-81cf-480b-8b11-ff3c510b6501 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:37.940824 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 587/1167] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:37.946118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbf6ef-a685-4d86-b086-017568b2b7c9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:37.946530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbf6ef-a685-4d86-b086-017568b2b7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:37.946717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbf6ef-a685-4d86-b086-017568b2b7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:37.963308 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-95fbf6ef-a685-4d86-b086-017568b2b7c9 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:37.964483 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-95fbf6ef-a685-4d86-b086-017568b2b7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:37.975636 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:38.031384 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 581/1168] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:47:38.055576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3354bd-81cf-480b-8b11-ff3c510b6501 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:38.154954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3354bd-81cf-480b-8b11-ff3c510b6501 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:38.179749 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:38.190472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d8f09-7487-47f3-90c3-d3e174b2b135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:38.191113 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d8f09-7487-47f3-90c3-d3e174b2b135 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:38.191391 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d8f09-7487-47f3-90c3-d3e174b2b135 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:38.191661 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d8f09-7487-47f3-90c3-d3e174b2b135 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:38.192382 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d8f09-7487-47f3-90c3-d3e174b2b135 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:38.193736 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d8f09-7487-47f3-90c3-d3e174b2b135 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:38.208151 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 588/1169] 10.222.0.70 () {60 vars in 1138 bytes} [Thu Jun 16 18:47:38 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:47:40.379978 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da50d12-cf63-4b8b-a41c-d0ef0bbb3dd3 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:40.380481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da50d12-cf63-4b8b-a41c-d0ef0bbb3dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:40.380481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da50d12-cf63-4b8b-a41c-d0ef0bbb3dd3 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:40.381816 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 582/1170] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:40.387185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f94b8-f1d2-4ce2-bff6-967c49d3c8ae None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:40.387587 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f94b8-f1d2-4ce2-bff6-967c49d3c8ae None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:40.387752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f94b8-f1d2-4ce2-bff6-967c49d3c8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:40.405769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ad2f94b8-f1d2-4ce2-bff6-967c49d3c8ae None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:40.407054 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ad2f94b8-f1d2-4ce2-bff6-967c49d3c8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:40.472564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 589/1171] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:47:40.497153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da50d12-cf63-4b8b-a41c-d0ef0bbb3dd3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:40.595210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da50d12-cf63-4b8b-a41c-d0ef0bbb3dd3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:40.629364 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7acc2-6326-4140-94db-3e3e5a1c753c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:40.630263 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7acc2-6326-4140-94db-3e3e5a1c753c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:40.630496 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7acc2-6326-4140-94db-3e3e5a1c753c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:40.630804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7acc2-6326-4140-94db-3e3e5a1c753c None admin] PATH_INFO: `/v3/domains/ba490c814c8b443ab337ef0a393a29f5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:40.641234 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7acc2-6326-4140-94db-3e3e5a1c753c None admin] RBAC: Authorizing `identity:get_domain(domain_id=ba490c814c8b443ab337ef0a393a29f5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:40.642776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7acc2-6326-4140-94db-3e3e5a1c753c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:40.644403 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 583/1172] 10.222.0.70 () {56 vars in 1183 bytes} [Thu Jun 16 18:47:40 2022] GET /identity/v3/domains/ba490c814c8b443ab337ef0a393a29f5 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:40.650340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f94b8-f1d2-4ce2-bff6-967c49d3c8ae None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:40.746038 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f94b8-f1d2-4ce2-bff6-967c49d3c8ae None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:40.760142 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:40.780720 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4bd3e-1ca3-4a9c-acbe-0ee3d62450ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:40.781366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4bd3e-1ca3-4a9c-acbe-0ee3d62450ae None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:40.781648 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4bd3e-1ca3-4a9c-acbe-0ee3d62450ae None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:40.781917 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4bd3e-1ca3-4a9c-acbe-0ee3d62450ae None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:40.782647 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4bd3e-1ca3-4a9c-acbe-0ee3d62450ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:40.784718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4bd3e-1ca3-4a9c-acbe-0ee3d62450ae None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:40.810970 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 590/1173] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Jun 16 18:47:40 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:47:43.016211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0b864-f574-4bb0-8a40-d06883c64f35 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:43.016738 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0b864-f574-4bb0-8a40-d06883c64f35 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:43.016738 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0b864-f574-4bb0-8a40-d06883c64f35 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:43.018884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 584/1174] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:43.024181 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff32ef9-d10d-4464-b2eb-4b31e82b351b None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:43.024599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff32ef9-d10d-4464-b2eb-4b31e82b351b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:43.024746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff32ef9-d10d-4464-b2eb-4b31e82b351b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:43.041837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-bff32ef9-d10d-4464-b2eb-4b31e82b351b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:43.043066 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-bff32ef9-d10d-4464-b2eb-4b31e82b351b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:43.109722 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 591/1175] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:47:43.135056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0b864-f574-4bb0-8a40-d06883c64f35 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:43.145510 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:43.237607 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0b864-f574-4bb0-8a40-d06883c64f35 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:43.271089 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3111b59-9922-49cc-8f67-716fc4d34f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:43.271940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3111b59-9922-49cc-8f67-716fc4d34f54 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:43.272221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3111b59-9922-49cc-8f67-716fc4d34f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:43.272495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3111b59-9922-49cc-8f67-716fc4d34f54 None admin] PATH_INFO: `/v3/domains/ba490c814c8b443ab337ef0a393a29f5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:43.273656 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3111b59-9922-49cc-8f67-716fc4d34f54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ba490c814c8b443ab337ef0a393a29f5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:43.275105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3111b59-9922-49cc-8f67-716fc4d34f54 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:43.276701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 585/1176] 10.222.0.70 () {56 vars in 1183 bytes} [Thu Jun 16 18:47:43 2022] GET /identity/v3/domains/ba490c814c8b443ab337ef0a393a29f5 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:43.282501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff32ef9-d10d-4464-b2eb-4b31e82b351b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:43.373721 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:43.380346 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff32ef9-d10d-4464-b2eb-4b31e82b351b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:43.414014 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e544fd-4444-4251-86c2-f3c1cc321930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:43.414697 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e544fd-4444-4251-86c2-f3c1cc321930 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:43.414976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e544fd-4444-4251-86c2-f3c1cc321930 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:43.415248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e544fd-4444-4251-86c2-f3c1cc321930 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:43.415983 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e544fd-4444-4251-86c2-f3c1cc321930 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:43.417370 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e544fd-4444-4251-86c2-f3c1cc321930 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:43.436830 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 592/1177] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:47:43 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:47:45.623880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda899c-8d9b-467e-91a6-032dcefafae2 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:45.624466 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda899c-8d9b-467e-91a6-032dcefafae2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:45.624466 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda899c-8d9b-467e-91a6-032dcefafae2 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:45.625784 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 586/1178] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:45.631467 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c034a1-7ce9-442d-9db3-bddca3d8370d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:45.631905 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c034a1-7ce9-442d-9db3-bddca3d8370d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:45.632023 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c034a1-7ce9-442d-9db3-bddca3d8370d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:45.649345 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-02c034a1-7ce9-442d-9db3-bddca3d8370d None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:45.650503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-02c034a1-7ce9-442d-9db3-bddca3d8370d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:45.715841 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 593/1179] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:47:45.739964 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda899c-8d9b-467e-91a6-032dcefafae2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:45.818630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:45.843663 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda899c-8d9b-467e-91a6-032dcefafae2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:45.877403 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f2a68-2c0e-4bb0-b26e-be8b78635644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:45.878116 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f2a68-2c0e-4bb0-b26e-be8b78635644 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:45.878391 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f2a68-2c0e-4bb0-b26e-be8b78635644 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:45.878688 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f2a68-2c0e-4bb0-b26e-be8b78635644 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:45.888870 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f2a68-2c0e-4bb0-b26e-be8b78635644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:47:45.896863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-3d5f2a68-2c0e-4bb0-b26e-be8b78635644 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:47:45.896863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:45.896863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:45.896863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:45.896863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:45.896863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:45.896863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:45.896863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:45.896863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:45.896863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:45.896863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:45.896863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:45.896863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:47:45.896863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:47:45.896863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:47:45.896863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:47:45.897959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:47:45.897959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:47:45.897959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:47:45.897959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:47:45.897959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:45.897959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:45.897959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:45.897959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:45.897959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:45.897959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:45.897959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:45.897959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:45.897959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:45.897959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:45.897959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:45.897959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:45.899288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:45.899288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:45.899288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:45.899288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:45.899288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:47:45.899288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:47:45.899288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:47:45.899288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:47:45.899288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:47:45.899288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:47:45.899288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:47:45.899288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:47:45.899288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 587/1180] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:47:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:47:45.903990 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c034a1-7ce9-442d-9db3-bddca3d8370d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:45.982966 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:46.006591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c034a1-7ce9-442d-9db3-bddca3d8370d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:46.042485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52a57d-9f5e-4040-a537-e08d65bc124d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:46.043179 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52a57d-9f5e-4040-a537-e08d65bc124d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:46.043465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52a57d-9f5e-4040-a537-e08d65bc124d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:46.043741 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52a57d-9f5e-4040-a537-e08d65bc124d None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:46.044478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52a57d-9f5e-4040-a537-e08d65bc124d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:46.044785 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52a57d-9f5e-4040-a537-e08d65bc124d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:46.046156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52a57d-9f5e-4040-a537-e08d65bc124d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:46.054787 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 594/1181] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:47:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:46.060811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f2a68-2c0e-4bb0-b26e-be8b78635644 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:46.158686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f2a68-2c0e-4bb0-b26e-be8b78635644 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:46.192587 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2fe818-f03c-45c6-b19d-1731c929cb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:46.193559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fe818-f03c-45c6-b19d-1731c929cb90 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:46.193771 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fe818-f03c-45c6-b19d-1731c929cb90 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:46.194047 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fe818-f03c-45c6-b19d-1731c929cb90 None admin] PATH_INFO: `/v3/users/b7cdb4d254f54352ae2feec16586dd10` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:46.195953 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:46.208262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2fe818-f03c-45c6-b19d-1731c929cb90 None admin] RBAC: Authorizing `identity:get_user(user_id=b7cdb4d254f54352ae2feec16586dd10)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:46.209693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2fe818-f03c-45c6-b19d-1731c929cb90 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:46.211382 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 588/1182] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:47:46 2022] GET /identity/v3/users/b7cdb4d254f54352ae2feec16586dd10 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:46.217159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52a57d-9f5e-4040-a537-e08d65bc124d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:46.317105 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52a57d-9f5e-4040-a537-e08d65bc124d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:46.349595 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff776506-d12a-4f31-8b79-3dc6f46263ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:46.350232 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff776506-d12a-4f31-8b79-3dc6f46263ae None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:46.350518 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff776506-d12a-4f31-8b79-3dc6f46263ae None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:46.350838 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff776506-d12a-4f31-8b79-3dc6f46263ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:46.365262 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:46.417360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff776506-d12a-4f31-8b79-3dc6f46263ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:46.418765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff776506-d12a-4f31-8b79-3dc6f46263ae None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:46.513669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 595/1183] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:47:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:47:46.520518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2fe818-f03c-45c6-b19d-1731c929cb90 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:46.617117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2fe818-f03c-45c6-b19d-1731c929cb90 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:46.650391 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed6338-8460-4b72-bdaa-46c6511f5d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:46.651267 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed6338-8460-4b72-bdaa-46c6511f5d5d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:46.651550 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed6338-8460-4b72-bdaa-46c6511f5d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:46.651824 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed6338-8460-4b72-bdaa-46c6511f5d5d None admin] PATH_INFO: `/v3/projects/748c85a63e5641d09c9b0fb08e67e555` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:46.652984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed6338-8460-4b72-bdaa-46c6511f5d5d None admin] RBAC: Authorizing `identity:get_project(project_id=748c85a63e5641d09c9b0fb08e67e555)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:46.654400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed6338-8460-4b72-bdaa-46c6511f5d5d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:46.656110 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 589/1184] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:47:46 2022] GET /identity/v3/projects/748c85a63e5641d09c9b0fb08e67e555 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:46.662443 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff776506-d12a-4f31-8b79-3dc6f46263ae None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:46.708683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:46.762670 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff776506-d12a-4f31-8b79-3dc6f46263ae None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:46.796632 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d8948-77c9-40c1-b086-511c03a1ba61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:46.797274 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d8948-77c9-40c1-b086-511c03a1ba61 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:46.797554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d8948-77c9-40c1-b086-511c03a1ba61 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:46.797829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d8948-77c9-40c1-b086-511c03a1ba61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:46.799242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d8948-77c9-40c1-b086-511c03a1ba61 None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=748c85a63e5641d09c9b0fb08e67e555, user.id=b7cdb4d254f54352ae2feec16586dd10) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:46.799562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d8948-77c9-40c1-b086-511c03a1ba61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:46.800975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d8948-77c9-40c1-b086-511c03a1ba61 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:46.806341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 596/1185] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:47:46 2022] GET /identity/v3/role_assignments?user.id=b7cdb4d254f54352ae2feec16586dd10&scope.project.id=748c85a63e5641d09c9b0fb08e67e555&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:48.884702 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575274a5-2474-4bb1-90bc-ebd6c029d8e8 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:48.885231 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575274a5-2474-4bb1-90bc-ebd6c029d8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:48.885231 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575274a5-2474-4bb1-90bc-ebd6c029d8e8 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:48.886642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 590/1186] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:48.892707 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c80a8-0603-4104-9e18-fe0cd18c8068 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:48.893157 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c80a8-0603-4104-9e18-fe0cd18c8068 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:48.893259 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c80a8-0603-4104-9e18-fe0cd18c8068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:48.910057 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-672c80a8-0603-4104-9e18-fe0cd18c8068 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:48.911262 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-672c80a8-0603-4104-9e18-fe0cd18c8068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:48.977286 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 597/1187] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:47:49.000008 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575274a5-2474-4bb1-90bc-ebd6c029d8e8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:49.061153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:49.102837 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575274a5-2474-4bb1-90bc-ebd6c029d8e8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:49.136140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bf875-6679-4c64-8a58-94a3f6c50a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:49.136848 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bf875-6679-4c64-8a58-94a3f6c50a86 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:49.137131 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bf875-6679-4c64-8a58-94a3f6c50a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:49.137406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bf875-6679-4c64-8a58-94a3f6c50a86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:49.148467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644bf875-6679-4c64-8a58-94a3f6c50a86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:47:49.156147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-644bf875-6679-4c64-8a58-94a3f6c50a86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:47:49.156147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:49.156147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:49.156147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:49.156147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:49.156147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:49.156147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:49.156147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:49.156147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:49.156147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:49.156147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:49.156147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:49.156147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:47:49.156147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:47:49.156147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:47:49.156147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:47:49.157332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:47:49.157332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:47:49.157332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:47:49.157332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:47:49.157332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:49.157332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:49.157332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:49.157332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:49.157332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:49.157332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:49.157332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:49.157332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:49.157332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:49.157332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:49.157332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:49.157332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:49.158487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:49.158487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:49.158487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:49.158487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:49.158487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:47:49.158487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:47:49.158487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:47:49.158487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:47:49.158487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:47:49.158487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:47:49.158487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:47:49.158487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:47:49.158487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 591/1188] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:47:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:47:49.163718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c80a8-0603-4104-9e18-fe0cd18c8068 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:49.203820 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:49.262426 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c80a8-0603-4104-9e18-fe0cd18c8068 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:49.296248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76bfd03-2f92-448a-b80d-44533ad8051c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:49.296908 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bfd03-2f92-448a-b80d-44533ad8051c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:49.297192 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bfd03-2f92-448a-b80d-44533ad8051c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:49.297471 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bfd03-2f92-448a-b80d-44533ad8051c None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:49.298316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76bfd03-2f92-448a-b80d-44533ad8051c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:49.298653 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76bfd03-2f92-448a-b80d-44533ad8051c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:49.300025 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76bfd03-2f92-448a-b80d-44533ad8051c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:49.307954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 598/1189] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:47:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:49.314184 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bf875-6679-4c64-8a58-94a3f6c50a86 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:49.413481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bf875-6679-4c64-8a58-94a3f6c50a86 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:49.433228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:49.447970 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee171f-12f2-47c6-867b-88cc5b9e62ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:49.448841 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee171f-12f2-47c6-867b-88cc5b9e62ef None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:49.449128 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee171f-12f2-47c6-867b-88cc5b9e62ef None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:49.449406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee171f-12f2-47c6-867b-88cc5b9e62ef None admin] PATH_INFO: `/v3/users/b7cdb4d254f54352ae2feec16586dd10` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:49.450611 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee171f-12f2-47c6-867b-88cc5b9e62ef None admin] RBAC: Authorizing `identity:get_user(user_id=b7cdb4d254f54352ae2feec16586dd10)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:49.452038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee171f-12f2-47c6-867b-88cc5b9e62ef None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:49.453682 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 592/1190] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:47:49 2022] GET /identity/v3/users/b7cdb4d254f54352ae2feec16586dd10 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:49.459808 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76bfd03-2f92-448a-b80d-44533ad8051c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:49.555898 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:49.559206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76bfd03-2f92-448a-b80d-44533ad8051c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:49.592239 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ade2d-5283-4d3e-84bf-7335f411cb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:49.593080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ade2d-5283-4d3e-84bf-7335f411cb83 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:49.593358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ade2d-5283-4d3e-84bf-7335f411cb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:49.593628 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ade2d-5283-4d3e-84bf-7335f411cb83 None admin] PATH_INFO: `/v3/projects/748c85a63e5641d09c9b0fb08e67e555` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:49.594856 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ade2d-5283-4d3e-84bf-7335f411cb83 None admin] RBAC: Authorizing `identity:get_project(project_id=748c85a63e5641d09c9b0fb08e67e555)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:49.596278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ade2d-5283-4d3e-84bf-7335f411cb83 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:49.597929 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 599/1191] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:47:49 2022] GET /identity/v3/projects/748c85a63e5641d09c9b0fb08e67e555 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:49.603858 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee171f-12f2-47c6-867b-88cc5b9e62ef None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:49.708074 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee171f-12f2-47c6-867b-88cc5b9e62ef None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:49.740938 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec997e1-64a1-48b2-b516-6cd116ee37bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:49.741813 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec997e1-64a1-48b2-b516-6cd116ee37bb None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:49.742090 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec997e1-64a1-48b2-b516-6cd116ee37bb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:49.742361 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec997e1-64a1-48b2-b516-6cd116ee37bb None admin] PATH_INFO: `/v3/projects/748c85a63e5641d09c9b0fb08e67e555/users/b7cdb4d254f54352ae2feec16586dd10/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:49.744376 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec997e1-64a1-48b2-b516-6cd116ee37bb None admin] RBAC: Authorizing `identity:create_grant(project_id=748c85a63e5641d09c9b0fb08e67e555, user_id=b7cdb4d254f54352ae2feec16586dd10, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:49.745797 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec997e1-64a1-48b2-b516-6cd116ee37bb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:49.759428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 593/1192] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:47:49 2022] PUT /identity/v3/projects/748c85a63e5641d09c9b0fb08e67e555/users/b7cdb4d254f54352ae2feec16586dd10/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:47:51.910895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244acbe3-8f6e-4cd2-9a92-38ed5a51c6cd None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:51.911432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244acbe3-8f6e-4cd2-9a92-38ed5a51c6cd None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:51.911432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244acbe3-8f6e-4cd2-9a92-38ed5a51c6cd None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:51.912808 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 600/1193] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:51.918147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63b1e3-6c43-40e4-8347-3b843fd9d75c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:51.918535 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63b1e3-6c43-40e4-8347-3b843fd9d75c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:51.918737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63b1e3-6c43-40e4-8347-3b843fd9d75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:51.935166 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-cb63b1e3-6c43-40e4-8347-3b843fd9d75c None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:51.936317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cb63b1e3-6c43-40e4-8347-3b843fd9d75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:51.995062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:52.023611 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 594/1194] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:47:52.047924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244acbe3-8f6e-4cd2-9a92-38ed5a51c6cd None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:52.150227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244acbe3-8f6e-4cd2-9a92-38ed5a51c6cd None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:52.182682 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e0a37-9882-4a48-aef8-1ff3fc848dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:52.183461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e0a37-9882-4a48-aef8-1ff3fc848dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:52.183746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e0a37-9882-4a48-aef8-1ff3fc848dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:52.184025 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e0a37-9882-4a48-aef8-1ff3fc848dd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:52.195089 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e0a37-9882-4a48-aef8-1ff3fc848dd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:47:52.202717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-f94e0a37-9882-4a48-aef8-1ff3fc848dd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:47:52.202717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:47:52.202717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:47:52.202717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:47:52.202717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:47:52.202717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:47:52.202717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:47:52.202717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:47:52.202717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:47:52.202717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:47:52.202717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:47:52.202717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:47:52.202717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:47:52.202717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:47:52.202717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:47:52.202717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:47:52.203813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:47:52.203813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:47:52.203813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:47:52.203813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:47:52.203813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:47:52.203813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:47:52.203813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:47:52.203813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:47:52.203813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:47:52.203813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:47:52.203813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:47:52.203813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:47:52.203813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:47:52.203813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:47:52.203813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:47:52.203813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:47:52.205244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:47:52.205244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:47:52.205244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:47:52.205244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:47:52.205244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:47:52.205244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:47:52.205244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:47:52.205244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:47:52.205244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:47:52.205244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:47:52.205244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:47:52.205244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:47:52.205244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 601/1195] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:47:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:47:52.210527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63b1e3-6c43-40e4-8347-3b843fd9d75c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:52.310918 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63b1e3-6c43-40e4-8347-3b843fd9d75c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:52.346745 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e00be3-1d7a-4cd9-9327-a5cb1f06d95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:52.347417 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e00be3-1d7a-4cd9-9327-a5cb1f06d95f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:52.347702 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e00be3-1d7a-4cd9-9327-a5cb1f06d95f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:52.347978 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e00be3-1d7a-4cd9-9327-a5cb1f06d95f None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:52.348728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e00be3-1d7a-4cd9-9327-a5cb1f06d95f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:52.349034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e00be3-1d7a-4cd9-9327-a5cb1f06d95f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:52.350397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e00be3-1d7a-4cd9-9327-a5cb1f06d95f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:52.358831 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 595/1196] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:47:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:52.365001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e0a37-9882-4a48-aef8-1ff3fc848dd5 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:52.393282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:52.463577 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e0a37-9882-4a48-aef8-1ff3fc848dd5 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:52.496398 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d31f63-1833-4ea0-b2d7-32fc777d6369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:52.497250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d31f63-1833-4ea0-b2d7-32fc777d6369 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:52.497529 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d31f63-1833-4ea0-b2d7-32fc777d6369 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:52.497802 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d31f63-1833-4ea0-b2d7-32fc777d6369 None admin] PATH_INFO: `/v3/users/b7cdb4d254f54352ae2feec16586dd10` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:52.499008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d31f63-1833-4ea0-b2d7-32fc777d6369 None admin] RBAC: Authorizing `identity:get_user(user_id=b7cdb4d254f54352ae2feec16586dd10)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:52.500429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d31f63-1833-4ea0-b2d7-32fc777d6369 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:52.502667 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 602/1197] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Jun 16 18:47:52 2022] GET /identity/v3/users/b7cdb4d254f54352ae2feec16586dd10 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:52.508519 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e00be3-1d7a-4cd9-9327-a5cb1f06d95f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:52.549862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:52.608085 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e00be3-1d7a-4cd9-9327-a5cb1f06d95f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:52.642077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172ce23-c450-42c3-992e-cc8b45606484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:52.642751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172ce23-c450-42c3-992e-cc8b45606484 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:52.643038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172ce23-c450-42c3-992e-cc8b45606484 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:52.643313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172ce23-c450-42c3-992e-cc8b45606484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:52.709573 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6172ce23-c450-42c3-992e-cc8b45606484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:52.711047 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6172ce23-c450-42c3-992e-cc8b45606484 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:52.749927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:52.810279 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 596/1198] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:47:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:47:52.816860 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d31f63-1833-4ea0-b2d7-32fc777d6369 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:52.911148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:52.917388 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d31f63-1833-4ea0-b2d7-32fc777d6369 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:52.950787 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f69071-97fa-4baf-ac78-5c3e6f1b2187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:52.951643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f69071-97fa-4baf-ac78-5c3e6f1b2187 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:52.951926 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f69071-97fa-4baf-ac78-5c3e6f1b2187 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:52.952202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f69071-97fa-4baf-ac78-5c3e6f1b2187 None admin] PATH_INFO: `/v3/projects/748c85a63e5641d09c9b0fb08e67e555` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:52.953371 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f69071-97fa-4baf-ac78-5c3e6f1b2187 None admin] RBAC: Authorizing `identity:get_project(project_id=748c85a63e5641d09c9b0fb08e67e555)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:52.955638 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f69071-97fa-4baf-ac78-5c3e6f1b2187 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:52.957328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 603/1199] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Jun 16 18:47:52 2022] GET /identity/v3/projects/748c85a63e5641d09c9b0fb08e67e555 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:47:52.963381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172ce23-c450-42c3-992e-cc8b45606484 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:53.059785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172ce23-c450-42c3-992e-cc8b45606484 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:53.093419 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564966d-cb91-41fa-93d3-9b49e6c456c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:53.094063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564966d-cb91-41fa-93d3-9b49e6c456c3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:53.094347 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564966d-cb91-41fa-93d3-9b49e6c456c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:53.094704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564966d-cb91-41fa-93d3-9b49e6c456c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:53.095521 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564966d-cb91-41fa-93d3-9b49e6c456c3 None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=748c85a63e5641d09c9b0fb08e67e555, user.id=b7cdb4d254f54352ae2feec16586dd10) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:47:53.095834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564966d-cb91-41fa-93d3-9b49e6c456c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:53.097246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564966d-cb91-41fa-93d3-9b49e6c456c3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:53.102965 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 597/1200] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:47:52 2022] GET /identity/v3/role_assignments?user.id=b7cdb4d254f54352ae2feec16586dd10&scope.project.id=748c85a63e5641d09c9b0fb08e67e555&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:48:02.948467 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=18 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:03.100346 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:52.042139 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28230f0-5f69-4635-936b-36c81dcd8f8c None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:52.042780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28230f0-5f69-4635-936b-36c81dcd8f8c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:52.042872 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28230f0-5f69-4635-936b-36c81dcd8f8c None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:52.045763 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 604/1201] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:48:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:52.052222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59afcb4-b046-4c1f-9c71-94f69e8b67ee None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:52.052775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59afcb4-b046-4c1f-9c71-94f69e8b67ee None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:52.052775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59afcb4-b046-4c1f-9c71-94f69e8b67ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:52.077679 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a59afcb4-b046-4c1f-9c71-94f69e8b67ee None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:52.078917 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a59afcb4-b046-4c1f-9c71-94f69e8b67ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:52.218687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 598/1202] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:48:52.245209 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28230f0-5f69-4635-936b-36c81dcd8f8c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:52.403175 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28230f0-5f69-4635-936b-36c81dcd8f8c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:52.461817 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc11d48-253f-4d03-9711-fff8bbae50ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:52.462816 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc11d48-253f-4d03-9711-fff8bbae50ad None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:52.463101 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc11d48-253f-4d03-9711-fff8bbae50ad None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:52.463401 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc11d48-253f-4d03-9711-fff8bbae50ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:52.479466 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc11d48-253f-4d03-9711-fff8bbae50ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:52.481373 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc11d48-253f-4d03-9711-fff8bbae50ad None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:52.507840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-abc11d48-253f-4d03-9711-fff8bbae50ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:48:52.507840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:48:52.507840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:48:52.507840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:48:52.507840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:48:52.507840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:48:52.507840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:48:52.507840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:48:52.507840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:48:52.507840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:48:52.507840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:48:52.507840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:48:52.507840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:48:52.507840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:48:52.507840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:48:52.509283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:48:52.509283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:48:52.509283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:48:52.509283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:48:52.509283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:48:52.509283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:48:52.509283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:48:52.509283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:48:52.509283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:48:52.509283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:48:52.509283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:48:52.509283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:48:52.509283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:48:52.509283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:48:52.509283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:48:52.509283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:48:52.509283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:48:52.510771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:48:52.510771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:48:52.510771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:48:52.510771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:48:52.510771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 605/1203] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:48:52.516304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59afcb4-b046-4c1f-9c71-94f69e8b67ee None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:52.689105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59afcb4-b046-4c1f-9c71-94f69e8b67ee None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:52.729605 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:52.734095 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161344e4-ae52-4582-b86b-e103d13fcddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:52.734937 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161344e4-ae52-4582-b86b-e103d13fcddf None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:52.735239 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161344e4-ae52-4582-b86b-e103d13fcddf None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:52.735480 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161344e4-ae52-4582-b86b-e103d13fcddf None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:52.736263 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161344e4-ae52-4582-b86b-e103d13fcddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:48:52.736607 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161344e4-ae52-4582-b86b-e103d13fcddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:52.739069 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161344e4-ae52-4582-b86b-e103d13fcddf None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:52.758116 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 599/1204] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:48:52.764881 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc11d48-253f-4d03-9711-fff8bbae50ad None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:52.850789 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:52.928311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc11d48-253f-4d03-9711-fff8bbae50ad None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:52.987588 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9276ae2-3812-487a-bb57-3bd71abf9c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:52.988833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9276ae2-3812-487a-bb57-3bd71abf9c01 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:52.989328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9276ae2-3812-487a-bb57-3bd71abf9c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:52.989630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9276ae2-3812-487a-bb57-3bd71abf9c01 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:52.990401 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9276ae2-3812-487a-bb57-3bd71abf9c01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:52.991985 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9276ae2-3812-487a-bb57-3bd71abf9c01 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:53.024001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 606/1205] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Jun 16 18:48:52 2022] POST /identity/v3/users => generated 251 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:48:55.596979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8db7d-96ab-43a9-9e3d-5bdcc61afa35 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:55.597674 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8db7d-96ab-43a9-9e3d-5bdcc61afa35 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:55.597674 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8db7d-96ab-43a9-9e3d-5bdcc61afa35 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:55.602708 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 600/1206] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:55.605755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff4cbf-75b6-42f8-b6ed-e03ffea12814 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:55.607999 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff4cbf-75b6-42f8-b6ed-e03ffea12814 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:55.607999 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff4cbf-75b6-42f8-b6ed-e03ffea12814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:55.631360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-0cff4cbf-75b6-42f8-b6ed-e03ffea12814 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:55.632725 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0cff4cbf-75b6-42f8-b6ed-e03ffea12814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:55.721068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 607/1207] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:48:55.750111 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8db7d-96ab-43a9-9e3d-5bdcc61afa35 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:55.880149 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8db7d-96ab-43a9-9e3d-5bdcc61afa35 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:55.928475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c797ae-20fd-4b76-add3-c591072b1e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:55.930193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c797ae-20fd-4b76-add3-c591072b1e30 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:55.930193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c797ae-20fd-4b76-add3-c591072b1e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:55.930388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c797ae-20fd-4b76-add3-c591072b1e30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:55.952275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c797ae-20fd-4b76-add3-c591072b1e30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:48:55.961562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-d9c797ae-20fd-4b76-add3-c591072b1e30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:48:55.961562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:48:55.961562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:48:55.961562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:48:55.961562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:48:55.961562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:48:55.961562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:48:55.961562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:48:55.961562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:48:55.961562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:48:55.961562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:48:55.961562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:48:55.961562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:48:55.961562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:48:55.961562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:48:55.961562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:48:55.963067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:48:55.963067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:48:55.963067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:48:55.963067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:48:55.963067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:48:55.963067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:48:55.963067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:48:55.963067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:48:55.963067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:48:55.963067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:48:55.963067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:48:55.963067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:48:55.963067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:48:55.963067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:48:55.963067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:48:55.963067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:48:55.964328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:48:55.964328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:48:55.964328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:48:55.964328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:48:55.964328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:48:55.964328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:48:55.964328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:48:55.964328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:48:55.964328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:48:55.964328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:48:55.964328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:48:55.964328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:48:55.964328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 601/1208] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:48:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:48:55.969875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff4cbf-75b6-42f8-b6ed-e03ffea12814 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:56.023875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:56.103716 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff4cbf-75b6-42f8-b6ed-e03ffea12814 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:56.154994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbace4-1e67-42fe-8405-566cef778c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:56.155592 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbace4-1e67-42fe-8405-566cef778c08 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:56.155995 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbace4-1e67-42fe-8405-566cef778c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:56.156473 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbace4-1e67-42fe-8405-566cef778c08 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:56.157825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbace4-1e67-42fe-8405-566cef778c08 None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:48:56.158318 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbace4-1e67-42fe-8405-566cef778c08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:56.160329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbace4-1e67-42fe-8405-566cef778c08 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:56.170999 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 608/1209] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:48:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:48:56.179063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c797ae-20fd-4b76-add3-c591072b1e30 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:56.238562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:56.319546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c797ae-20fd-4b76-add3-c591072b1e30 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:56.363577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121668e4-82a4-48d9-8af7-ededd914b6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:56.364792 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121668e4-82a4-48d9-8af7-ededd914b6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:56.365328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121668e4-82a4-48d9-8af7-ededd914b6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:56.365804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121668e4-82a4-48d9-8af7-ededd914b6f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:56.381054 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121668e4-82a4-48d9-8af7-ededd914b6f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:56.383012 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121668e4-82a4-48d9-8af7-ededd914b6f1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:56.407530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-121668e4-82a4-48d9-8af7-ededd914b6f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:48:56.407530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:48:56.407530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:48:56.407530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:48:56.407530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:48:56.407530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:48:56.407530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:48:56.407530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:48:56.407530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:48:56.407530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:48:56.407530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:48:56.407530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:48:56.407530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:48:56.407530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:48:56.407530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:48:56.408508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:48:56.408508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:48:56.408508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:48:56.408508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:48:56.408508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:48:56.408508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:48:56.408508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:48:56.408508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:48:56.408508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:48:56.408508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:48:56.408508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:48:56.408508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:48:56.408508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:48:56.408508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:48:56.408508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:48:56.408508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:48:56.408508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:48:56.409436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:48:56.409436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:48:56.409436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:48:56.409436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:48:56.409436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 602/1210] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:48:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:48:56.423415 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbace4-1e67-42fe-8405-566cef778c08 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:56.565853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:56.573178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbace4-1e67-42fe-8405-566cef778c08 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:56.621008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19751bb9-4f15-4596-bfd8-8dc3e4cad1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:56.621860 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19751bb9-4f15-4596-bfd8-8dc3e4cad1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:56.622081 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19751bb9-4f15-4596-bfd8-8dc3e4cad1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:56.622440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19751bb9-4f15-4596-bfd8-8dc3e4cad1d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:56.623197 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19751bb9-4f15-4596-bfd8-8dc3e4cad1d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:48:56.623660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19751bb9-4f15-4596-bfd8-8dc3e4cad1d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:56.625348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19751bb9-4f15-4596-bfd8-8dc3e4cad1d2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:56.641011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 609/1211] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:48:56.651532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121668e4-82a4-48d9-8af7-ededd914b6f1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:56.794647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121668e4-82a4-48d9-8af7-ededd914b6f1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:56.797182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:56.850821 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af2873-69be-4ca4-a671-f3d857c442b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:56.851822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af2873-69be-4ca4-a671-f3d857c442b0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:56.852236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af2873-69be-4ca4-a671-f3d857c442b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:56.852580 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af2873-69be-4ca4-a671-f3d857c442b0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:56.866349 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af2873-69be-4ca4-a671-f3d857c442b0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:56.868342 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af2873-69be-4ca4-a671-f3d857c442b0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:56.892604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-85af2873-69be-4ca4-a671-f3d857c442b0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:48:56.892604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:48:56.892604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:48:56.892604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:48:56.892604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:48:56.892604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:48:56.892604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:48:56.892604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:48:56.892604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:48:56.892604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:48:56.892604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:48:56.892604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:48:56.892604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:48:56.892604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:48:56.892604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:48:56.892604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:48:56.900059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:48:56.901072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:48:56.901072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:48:56.901072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:48:56.901072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:48:56.901072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:48:56.901072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:48:56.901072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:48:56.901072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:48:56.901072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:48:56.901072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:48:56.901072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 603/1212] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:48:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:48:56.901714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19751bb9-4f15-4596-bfd8-8dc3e4cad1d2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:57.046332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19751bb9-4f15-4596-bfd8-8dc3e4cad1d2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:57.099752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7cd03-3edc-4c23-b32c-9c106a068b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:57.100582 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7cd03-3edc-4c23-b32c-9c106a068b7b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:57.101159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7cd03-3edc-4c23-b32c-9c106a068b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:57.101282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7cd03-3edc-4c23-b32c-9c106a068b7b None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:57.102060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7cd03-3edc-4c23-b32c-9c106a068b7b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:48:57.102374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7cd03-3edc-4c23-b32c-9c106a068b7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:57.103862 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7cd03-3edc-4c23-b32c-9c106a068b7b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:57.126847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 610/1213] 10.222.0.70 () {56 vars in 1143 bytes} [Thu Jun 16 18:48:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:48:57.133140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af2873-69be-4ca4-a671-f3d857c442b0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:57.268630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af2873-69be-4ca4-a671-f3d857c442b0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:57.328946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8ca36-7187-4afc-8b05-88b21e2a9005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:57.328946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8ca36-7187-4afc-8b05-88b21e2a9005 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:57.328946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8ca36-7187-4afc-8b05-88b21e2a9005 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:57.328946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8ca36-7187-4afc-8b05-88b21e2a9005 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:57.366104 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:57.424455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8ca36-7187-4afc-8b05-88b21e2a9005 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:57.426244 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8ca36-7187-4afc-8b05-88b21e2a9005 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:57.593521 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 604/1214] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:48:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:48:57.593840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7cd03-3edc-4c23-b32c-9c106a068b7b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:57.618056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:57.739122 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7cd03-3edc-4c23-b32c-9c106a068b7b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:57.801708 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ef2d0-3324-4bea-8bfb-f2d10816a069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:57.803049 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ef2d0-3324-4bea-8bfb-f2d10816a069 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:57.803462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ef2d0-3324-4bea-8bfb-f2d10816a069 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:57.803832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ef2d0-3324-4bea-8bfb-f2d10816a069 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:57.822802 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ef2d0-3324-4bea-8bfb-f2d10816a069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:57.824399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ef2d0-3324-4bea-8bfb-f2d10816a069 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:57.848438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-b60ef2d0-3324-4bea-8bfb-f2d10816a069 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:48:57.848438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:48:57.848438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:48:57.848438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:48:57.848438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:48:57.848438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:48:57.848438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:48:57.848438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:48:57.848438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:48:57.848438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:48:57.848438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:48:57.848438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:48:57.848438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:48:57.848438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:48:57.848438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:48:57.855289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:48:57.855289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:48:57.855289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:48:57.855289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:48:57.855289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:48:57.855289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:48:57.855289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:48:57.855289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:48:57.855289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:48:57.855289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:48:57.855289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:48:57.855289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:48:57.855289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:48:57.855289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:48:57.855289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:48:57.855289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:48:57.855289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:48:57.856601 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:48:57.856601 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:48:57.856601 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:48:57.856601 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:48:57.856601 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 611/1215] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:48:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:48:57.860313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8ca36-7187-4afc-8b05-88b21e2a9005 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:57.941145 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:58.014323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8ca36-7187-4afc-8b05-88b21e2a9005 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:58.068405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d212f16-2e03-4514-9548-c3a69269cc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:58.069227 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d212f16-2e03-4514-9548-c3a69269cc85 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:58.069533 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d212f16-2e03-4514-9548-c3a69269cc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:58.069801 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d212f16-2e03-4514-9548-c3a69269cc85 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:58.070522 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d212f16-2e03-4514-9548-c3a69269cc85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:48:58.070930 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d212f16-2e03-4514-9548-c3a69269cc85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:58.072561 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d212f16-2e03-4514-9548-c3a69269cc85 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:58.087433 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 605/1216] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:48:58.094693 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ef2d0-3324-4bea-8bfb-f2d10816a069 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:58.193360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:58.245424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ef2d0-3324-4bea-8bfb-f2d10816a069 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:58.289262 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931a973-b7f2-4047-a9ad-e763aecd8db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:58.290077 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931a973-b7f2-4047-a9ad-e763aecd8db0 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:58.290361 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931a973-b7f2-4047-a9ad-e763aecd8db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:58.290671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931a973-b7f2-4047-a9ad-e763aecd8db0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:58.298952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931a973-b7f2-4047-a9ad-e763aecd8db0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:58.300412 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931a973-b7f2-4047-a9ad-e763aecd8db0 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:58.316717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-e931a973-b7f2-4047-a9ad-e763aecd8db0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:48:58.316717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:48:58.316717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:48:58.316717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:48:58.316717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:48:58.316717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:48:58.316717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:48:58.316717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:48:58.316717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:48:58.316717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:48:58.316717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:48:58.316717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:48:58.316717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:48:58.316717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:48:58.316717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:48:58.318022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:48:58.318022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:48:58.318022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:48:58.318022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:48:58.318022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:48:58.318022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:48:58.318022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:48:58.318022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:48:58.318022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:48:58.318022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:48:58.318022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:48:58.318022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:48:58.318022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:48:58.318022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:48:58.318022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:48:58.318022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:48:58.318022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:48:58.318986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:48:58.318986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:48:58.318986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:48:58.318986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:48:58.318986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:48:58.318986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:48:58.318986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:48:58.318986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:48:58.318986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 612/1217] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:48:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:48:58.324560 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d212f16-2e03-4514-9548-c3a69269cc85 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:58.468592 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d212f16-2e03-4514-9548-c3a69269cc85 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:58.492536 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:58.508976 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0167be-c68a-47c0-bed9-766c9fa1e9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:58.509932 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0167be-c68a-47c0-bed9-766c9fa1e9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:58.510353 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0167be-c68a-47c0-bed9-766c9fa1e9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:58.510828 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0167be-c68a-47c0-bed9-766c9fa1e9c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:58.511739 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0167be-c68a-47c0-bed9-766c9fa1e9c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:48:58.512113 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0167be-c68a-47c0-bed9-766c9fa1e9c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:58.514610 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0167be-c68a-47c0-bed9-766c9fa1e9c4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:58.530790 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 606/1218] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:48:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:48:58.536586 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931a973-b7f2-4047-a9ad-e763aecd8db0 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:58.660386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931a973-b7f2-4047-a9ad-e763aecd8db0 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:58.700030 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:58.712372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c4ca9-3a5f-4063-a6bb-e663ef39f5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:58.713231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c4ca9-3a5f-4063-a6bb-e663ef39f5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:58.713619 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c4ca9-3a5f-4063-a6bb-e663ef39f5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:58.714047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c4ca9-3a5f-4063-a6bb-e663ef39f5c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:58.715186 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c4ca9-3a5f-4063-a6bb-e663ef39f5c5 None admin] RBAC: Adding query filter params (role.id=c123034c11cc4d9ea6cab00e05bdcd53, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=880d0299c38c4eb897c669a512e599be) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:48:58.715623 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c4ca9-3a5f-4063-a6bb-e663ef39f5c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:58.718123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c4ca9-3a5f-4063-a6bb-e663ef39f5c5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:58.738953 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 613/1219] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:48:58 2022] GET /identity/v3/role_assignments?user.id=880d0299c38c4eb897c669a512e599be&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=c123034c11cc4d9ea6cab00e05bdcd53 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:01.153607 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20422576-8585-461d-bcdf-555260c1a4b8 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:01.154251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20422576-8585-461d-bcdf-555260c1a4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:01.154251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20422576-8585-461d-bcdf-555260c1a4b8 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:01.155793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 607/1220] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:49:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:49:01.162289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc40080-a6ac-4158-95b3-c1baf9290e01 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:01.162825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc40080-a6ac-4158-95b3-c1baf9290e01 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:01.163079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc40080-a6ac-4158-95b3-c1baf9290e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:01.182801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-fdc40080-a6ac-4158-95b3-c1baf9290e01 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:01.184048 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-fdc40080-a6ac-4158-95b3-c1baf9290e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:01.267274 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 614/1221] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:49:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:49:01.295788 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20422576-8585-461d-bcdf-555260c1a4b8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:01.435832 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20422576-8585-461d-bcdf-555260c1a4b8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:01.478886 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe38db-7137-46de-bf5b-363011ef7027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:01.479702 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe38db-7137-46de-bf5b-363011ef7027 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:01.479987 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe38db-7137-46de-bf5b-363011ef7027 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:01.480253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe38db-7137-46de-bf5b-363011ef7027 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:01.492036 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe38db-7137-46de-bf5b-363011ef7027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:49:01.500109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-68fe38db-7137-46de-bf5b-363011ef7027 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:49:01.500109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:01.500109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:01.500109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:01.500109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:01.500109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:01.500109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:01.500109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:01.500109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:01.500109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:01.500109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:01.500109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:01.500109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:49:01.500109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:49:01.500109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:49:01.500109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:49:01.501144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:49:01.501144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:49:01.501144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:49:01.501144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:49:01.501144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:01.501144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:01.501144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:01.501144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:01.501144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:01.501144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:01.501144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:01.501144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:01.501144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:01.501144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:01.501144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:01.501144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:01.502023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:01.502023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:01.502023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:01.502023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:01.502023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:49:01.502023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:49:01.502023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:49:01.502023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:49:01.502023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:49:01.502023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:49:01.502023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:49:01.502023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:01.502023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 608/1222] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:49:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:01.508511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc40080-a6ac-4158-95b3-c1baf9290e01 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:01.645944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc40080-a6ac-4158-95b3-c1baf9290e01 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:01.680310 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:01.696190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af014f04-72b0-46fd-ae40-e81c2508d8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:01.697029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af014f04-72b0-46fd-ae40-e81c2508d8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:01.697400 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af014f04-72b0-46fd-ae40-e81c2508d8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:01.697764 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af014f04-72b0-46fd-ae40-e81c2508d8a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:01.698518 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af014f04-72b0-46fd-ae40-e81c2508d8a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:01.698878 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af014f04-72b0-46fd-ae40-e81c2508d8a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:01.700793 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af014f04-72b0-46fd-ae40-e81c2508d8a2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:01.717896 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 615/1223] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:49:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:01.726458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe38db-7137-46de-bf5b-363011ef7027 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:01.782830 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:01.844420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe38db-7137-46de-bf5b-363011ef7027 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:01.898464 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc9bdc2-d658-4c10-917b-6832be925c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:01.899841 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9bdc2-d658-4c10-917b-6832be925c75 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:01.900456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9bdc2-d658-4c10-917b-6832be925c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:01.900935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9bdc2-d658-4c10-917b-6832be925c75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:01.925324 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc9bdc2-d658-4c10-917b-6832be925c75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:01.928190 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc9bdc2-d658-4c10-917b-6832be925c75 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:01.963574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-4bc9bdc2-d658-4c10-917b-6832be925c75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:01.963574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:01.963574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:01.963574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:01.963574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:01.963574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:01.963574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:01.963574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:01.963574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:01.963574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:01.963574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:01.963574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:01.963574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:49:01.963574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:49:01.963574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:49:01.964737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:49:01.964737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:01.964737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:01.964737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:01.964737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:01.964737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:01.964737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:01.964737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:01.964737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:01.964737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:01.964737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:01.964737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:01.964737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:01.964737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:01.964737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:01.964737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:01.964737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:01.965891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:49:01.965891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:49:01.965891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:01.965891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:01.966941 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 609/1224] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:49:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:01.973005 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af014f04-72b0-46fd-ae40-e81c2508d8a2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:02.116444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af014f04-72b0-46fd-ae40-e81c2508d8a2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:02.158282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee1923-51c9-4e97-ba4b-ae687cde8500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:02.158909 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee1923-51c9-4e97-ba4b-ae687cde8500 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:02.159188 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee1923-51c9-4e97-ba4b-ae687cde8500 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:02.159461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee1923-51c9-4e97-ba4b-ae687cde8500 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:02.160179 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee1923-51c9-4e97-ba4b-ae687cde8500 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:02.160482 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee1923-51c9-4e97-ba4b-ae687cde8500 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:02.161839 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee1923-51c9-4e97-ba4b-ae687cde8500 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:02.176506 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 616/1225] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:02.182722 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc9bdc2-d658-4c10-917b-6832be925c75 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:02.301800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:02.308853 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc9bdc2-d658-4c10-917b-6832be925c75 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:02.353954 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa43a43-9d93-4600-afe3-58ed535ba968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:02.354816 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa43a43-9d93-4600-afe3-58ed535ba968 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:02.355205 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa43a43-9d93-4600-afe3-58ed535ba968 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:02.355428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa43a43-9d93-4600-afe3-58ed535ba968 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:02.364585 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa43a43-9d93-4600-afe3-58ed535ba968 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:02.366035 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa43a43-9d93-4600-afe3-58ed535ba968 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:02.381044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-1fa43a43-9d93-4600-afe3-58ed535ba968 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:49:02.381044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:02.381044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:02.381044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:02.381044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:02.381044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:02.381044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:02.381044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:02.381044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:02.381044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:02.381044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:02.381044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:02.381044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:49:02.381044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:49:02.381044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:49:02.381044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:02.382298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:02.383459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:02.383459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:02.383459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:49:02.383459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:49:02.383459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:49:02.383459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:49:02.383459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:49:02.383459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:49:02.383459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:49:02.383459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:02.383459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 610/1226] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:49:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:02.388487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee1923-51c9-4e97-ba4b-ae687cde8500 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:02.447397 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:02.510454 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee1923-51c9-4e97-ba4b-ae687cde8500 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:02.550682 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d720fa-5712-4b09-a721-a4ef9b31836f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:02.551541 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d720fa-5712-4b09-a721-a4ef9b31836f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:02.551842 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d720fa-5712-4b09-a721-a4ef9b31836f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:02.552300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d720fa-5712-4b09-a721-a4ef9b31836f None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:02.553214 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d720fa-5712-4b09-a721-a4ef9b31836f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:02.553552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d720fa-5712-4b09-a721-a4ef9b31836f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:02.555191 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d720fa-5712-4b09-a721-a4ef9b31836f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:02.577148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 617/1227] 10.222.0.70 () {56 vars in 1143 bytes} [Thu Jun 16 18:49:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:02.584003 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa43a43-9d93-4600-afe3-58ed535ba968 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:02.706978 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa43a43-9d93-4600-afe3-58ed535ba968 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:02.754638 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad2a1a-4cda-4e72-9a0c-aa2d7ab91b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:02.755433 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad2a1a-4cda-4e72-9a0c-aa2d7ab91b01 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:02.755705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad2a1a-4cda-4e72-9a0c-aa2d7ab91b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:02.756003 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad2a1a-4cda-4e72-9a0c-aa2d7ab91b01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:02.768187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad2a1a-4cda-4e72-9a0c-aa2d7ab91b01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:02.769635 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad2a1a-4cda-4e72-9a0c-aa2d7ab91b01 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:02.787700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-afad2a1a-4cda-4e72-9a0c-aa2d7ab91b01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:02.787700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:02.787700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:02.787700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:02.787700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:02.787700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:02.787700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:02.787700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:02.787700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:02.787700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:02.787700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:02.787700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:02.787700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:49:02.787700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:49:02.787700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:49:02.788710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:49:02.788710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:02.788710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:02.788710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:02.788710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:02.788710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:02.788710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:02.788710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:02.788710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:02.788710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:02.788710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:02.788710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:02.788710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:02.788710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:02.788710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:02.788710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:02.788710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:02.789632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:49:02.789632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:49:02.789632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:02.789632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:02.789632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 611/1228] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:49:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:02.795981 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d720fa-5712-4b09-a721-a4ef9b31836f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:02.926967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d720fa-5712-4b09-a721-a4ef9b31836f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:02.930483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:02.980300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05d320-bcfa-47a3-8a86-1b42634fcfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:02.981062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05d320-bcfa-47a3-8a86-1b42634fcfd8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:02.981318 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05d320-bcfa-47a3-8a86-1b42634fcfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:02.982835 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05d320-bcfa-47a3-8a86-1b42634fcfd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:02.983038 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05d320-bcfa-47a3-8a86-1b42634fcfd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:02.983606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05d320-bcfa-47a3-8a86-1b42634fcfd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:02.985467 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05d320-bcfa-47a3-8a86-1b42634fcfd8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:03.001326 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 618/1229] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:03.007421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad2a1a-4cda-4e72-9a0c-aa2d7ab91b01 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:03.018602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:03.147378 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad2a1a-4cda-4e72-9a0c-aa2d7ab91b01 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:03.191039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd2f33-77b1-4126-b8b7-7a0b6704f5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:03.191995 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd2f33-77b1-4126-b8b7-7a0b6704f5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:03.192345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd2f33-77b1-4126-b8b7-7a0b6704f5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:03.192760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd2f33-77b1-4126-b8b7-7a0b6704f5f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:03.201606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd2f33-77b1-4126-b8b7-7a0b6704f5f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:03.203299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd2f33-77b1-4126-b8b7-7a0b6704f5f3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:03.219618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-74fd2f33-77b1-4126-b8b7-7a0b6704f5f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:49:03.219618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:03.219618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:03.219618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:03.219618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:03.219618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:03.219618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:03.219618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:03.219618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:03.219618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:03.219618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:03.219618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:03.219618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:49:03.219618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:49:03.219618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:49:03.220701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:49:03.220701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:03.220701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:03.220701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:03.220701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:03.220701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:03.220701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:03.220701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:03.220701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:03.220701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:03.220701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:03.220701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:03.220701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:03.220701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:03.220701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:03.220701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:03.220701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:03.221776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:49:03.221776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:49:03.221776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:49:03.221776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:49:03.221776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:49:03.221776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:49:03.221776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:49:03.221776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:03.221776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 612/1230] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:49:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:03.227931 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05d320-bcfa-47a3-8a86-1b42634fcfd8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:03.371238 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05d320-bcfa-47a3-8a86-1b42634fcfd8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:03.416030 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc9142-6342-4f00-8f7e-9df09d3c124f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:03.416859 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc9142-6342-4f00-8f7e-9df09d3c124f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:03.417176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc9142-6342-4f00-8f7e-9df09d3c124f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:03.417626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc9142-6342-4f00-8f7e-9df09d3c124f None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:03.418396 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc9142-6342-4f00-8f7e-9df09d3c124f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:03.418774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc9142-6342-4f00-8f7e-9df09d3c124f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:03.420275 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc9142-6342-4f00-8f7e-9df09d3c124f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:03.437173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 619/1231] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:49:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:03.444082 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd2f33-77b1-4126-b8b7-7a0b6704f5f3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:03.518427 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:03.561105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd2f33-77b1-4126-b8b7-7a0b6704f5f3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:03.606527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdc87b-84fa-41fe-9c74-656a235ca0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:03.608433 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdc87b-84fa-41fe-9c74-656a235ca0f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:03.608739 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdc87b-84fa-41fe-9c74-656a235ca0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:03.609063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdc87b-84fa-41fe-9c74-656a235ca0f0 None admin] PATH_INFO: `/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/880d0299c38c4eb897c669a512e599be/roles/c123034c11cc4d9ea6cab00e05bdcd53` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:03.630142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bdc87b-84fa-41fe-9c74-656a235ca0f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=34185125e75d4aad8ad5382ab22ee8a2, user_id=880d0299c38c4eb897c669a512e599be, role_id=c123034c11cc4d9ea6cab00e05bdcd53)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:03.631527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bdc87b-84fa-41fe-9c74-656a235ca0f0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:03.649196 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 613/1232] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:49:03 2022] PUT /identity/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/880d0299c38c4eb897c669a512e599be/roles/c123034c11cc4d9ea6cab00e05bdcd53 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:49:05.990501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26059c1-4b31-4720-bd2a-a7cb2086f464 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:05.991454 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26059c1-4b31-4720-bd2a-a7cb2086f464 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:05.991454 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26059c1-4b31-4720-bd2a-a7cb2086f464 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:05.992586 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 620/1233] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:06.000356 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a77dc-0849-4cde-8bbe-1bdc59971a51 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:06.001112 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a77dc-0849-4cde-8bbe-1bdc59971a51 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:06.001112 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a77dc-0849-4cde-8bbe-1bdc59971a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:06.024058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b60a77dc-0849-4cde-8bbe-1bdc59971a51 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:06.025733 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b60a77dc-0849-4cde-8bbe-1bdc59971a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:06.116793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 614/1234] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:49:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:49:06.149241 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26059c1-4b31-4720-bd2a-a7cb2086f464 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:06.185517 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:06.279757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26059c1-4b31-4720-bd2a-a7cb2086f464 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:06.319772 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cca6c5-f330-407d-83a4-eba280fb6388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:06.320574 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cca6c5-f330-407d-83a4-eba280fb6388 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:06.320860 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cca6c5-f330-407d-83a4-eba280fb6388 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:06.321139 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cca6c5-f330-407d-83a4-eba280fb6388 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:06.333878 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cca6c5-f330-407d-83a4-eba280fb6388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:49:06.342994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-81cca6c5-f330-407d-83a4-eba280fb6388 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:49:06.342994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:06.342994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:06.342994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:06.342994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:06.342994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:06.342994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:06.342994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:06.342994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:06.342994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:06.342994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:06.342994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:06.342994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:49:06.342994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:49:06.342994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:49:06.342994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:49:06.344522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:49:06.344522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:49:06.344522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:49:06.344522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:49:06.344522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:06.344522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:06.344522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:06.344522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:06.344522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:06.344522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:06.344522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:06.344522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:06.344522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:06.344522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:06.344522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:06.344522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:06.345853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:06.345853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:06.345853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:06.345853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:06.345853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:49:06.345853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:49:06.345853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:49:06.345853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:49:06.345853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:49:06.345853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:49:06.345853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:49:06.345853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:49:06.345853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 621/1235] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Jun 16 18:49:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:06.350888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a77dc-0849-4cde-8bbe-1bdc59971a51 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:06.402427 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:06.461998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a77dc-0849-4cde-8bbe-1bdc59971a51 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:06.500980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e94d0e-7907-47fe-a149-801eda32ec2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:06.501733 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e94d0e-7907-47fe-a149-801eda32ec2e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:06.502016 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e94d0e-7907-47fe-a149-801eda32ec2e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:06.502291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e94d0e-7907-47fe-a149-801eda32ec2e None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:06.503070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e94d0e-7907-47fe-a149-801eda32ec2e None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:06.503373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e94d0e-7907-47fe-a149-801eda32ec2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:06.505038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e94d0e-7907-47fe-a149-801eda32ec2e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:06.516034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 615/1236] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Jun 16 18:49:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:06.521878 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cca6c5-f330-407d-83a4-eba280fb6388 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:06.626043 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:06.645841 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cca6c5-f330-407d-83a4-eba280fb6388 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:06.684817 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35021e93-2110-4707-92bc-35e075357e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:06.686818 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35021e93-2110-4707-92bc-35e075357e6d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:06.686818 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35021e93-2110-4707-92bc-35e075357e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:06.687171 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35021e93-2110-4707-92bc-35e075357e6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:06.698924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35021e93-2110-4707-92bc-35e075357e6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:06.701813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35021e93-2110-4707-92bc-35e075357e6d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:06.723826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-35021e93-2110-4707-92bc-35e075357e6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:06.723826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:06.723826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:06.723826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:06.723826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:06.723826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:06.723826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:06.723826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:06.723826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:06.723826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:06.723826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:06.723826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:06.723826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:49:06.723826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:49:06.723826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:49:06.726123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:49:06.726123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:06.726123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:06.726123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:06.726123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:06.726123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:06.726123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:06.726123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:06.726123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:06.726123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:06.726123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:06.726123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:06.726123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:06.726123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:06.726123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:06.726123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:06.726123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:06.727702 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:49:06.727702 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:49:06.727702 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:06.727702 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:49:06.727702 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 622/1237] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:49:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:06.732281 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e94d0e-7907-47fe-a149-801eda32ec2e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:06.883936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e94d0e-7907-47fe-a149-801eda32ec2e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:06.894809 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:06.927661 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3afa34-c892-4950-89f6-efc7407e974c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:06.928335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3afa34-c892-4950-89f6-efc7407e974c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:06.928619 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3afa34-c892-4950-89f6-efc7407e974c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:06.928900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3afa34-c892-4950-89f6-efc7407e974c None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:06.929629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3afa34-c892-4950-89f6-efc7407e974c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:06.929943 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3afa34-c892-4950-89f6-efc7407e974c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:06.931423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3afa34-c892-4950-89f6-efc7407e974c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:06.958497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 616/1238] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:49:06.959838 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35021e93-2110-4707-92bc-35e075357e6d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:07.084756 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35021e93-2110-4707-92bc-35e075357e6d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:07.125829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e7303-974a-4419-80d9-3b2b6d19508d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:07.126787 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e7303-974a-4419-80d9-3b2b6d19508d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:07.127174 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e7303-974a-4419-80d9-3b2b6d19508d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:07.127522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e7303-974a-4419-80d9-3b2b6d19508d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:07.131209 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:07.138827 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e7303-974a-4419-80d9-3b2b6d19508d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:07.140553 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e7303-974a-4419-80d9-3b2b6d19508d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:07.159242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ec3e7303-974a-4419-80d9-3b2b6d19508d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:49:07.159242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:07.159242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:07.159242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:07.159242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:07.159242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:07.159242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:07.159242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:07.159242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:07.159242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:07.159242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:07.159242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:07.159242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:49:07.159242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:49:07.159242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:49:07.159242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:07.161231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:07.162774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:07.162774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:07.162774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:49:07.162774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:49:07.162774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:49:07.162774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:49:07.162774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:49:07.162774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:49:07.162774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:49:07.162774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:49:07.165823 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 623/1239] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:49:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:49:07.166969 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3afa34-c892-4950-89f6-efc7407e974c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:07.307395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3afa34-c892-4950-89f6-efc7407e974c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:07.354790 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6833c8-ffce-46c0-a902-11e6720c5f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:07.355696 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6833c8-ffce-46c0-a902-11e6720c5f80 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:07.356880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6833c8-ffce-46c0-a902-11e6720c5f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:07.357621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6833c8-ffce-46c0-a902-11e6720c5f80 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:07.358848 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6833c8-ffce-46c0-a902-11e6720c5f80 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:07.359264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6833c8-ffce-46c0-a902-11e6720c5f80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:07.361089 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6833c8-ffce-46c0-a902-11e6720c5f80 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:07.392543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 617/1240] 10.222.0.70 () {56 vars in 1143 bytes} [Thu Jun 16 18:49:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:07.394047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e7303-974a-4419-80d9-3b2b6d19508d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:07.516844 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e7303-974a-4419-80d9-3b2b6d19508d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:07.567234 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ceefd-5aee-4791-93f0-e5e16d949454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:07.568288 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ceefd-5aee-4791-93f0-e5e16d949454 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:07.568745 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ceefd-5aee-4791-93f0-e5e16d949454 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:07.569119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ceefd-5aee-4791-93f0-e5e16d949454 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:07.661223 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:07.668216 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ceefd-5aee-4791-93f0-e5e16d949454 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:07.669798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ceefd-5aee-4791-93f0-e5e16d949454 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:07.807849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 624/1241] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:49:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:49:07.815100 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6833c8-ffce-46c0-a902-11e6720c5f80 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:07.854868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:07.938319 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6833c8-ffce-46c0-a902-11e6720c5f80 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:07.975844 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9cc6dc-60ac-4684-9a9f-49ee07a16c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:07.976626 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cc6dc-60ac-4684-9a9f-49ee07a16c4e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:07.976902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cc6dc-60ac-4684-9a9f-49ee07a16c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:07.977164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cc6dc-60ac-4684-9a9f-49ee07a16c4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:07.989650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9cc6dc-60ac-4684-9a9f-49ee07a16c4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:07.991299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9cc6dc-60ac-4684-9a9f-49ee07a16c4e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:08.015142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-2d9cc6dc-60ac-4684-9a9f-49ee07a16c4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:08.015142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:08.015142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:08.015142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:08.015142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:08.015142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:08.015142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:08.015142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:08.015142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:08.015142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:08.015142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:08.015142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:08.015142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:49:08.015142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:49:08.015142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:49:08.016497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:49:08.016497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:08.016497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:08.016497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:08.016497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:08.016497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:08.016497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:08.016497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:08.016497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:08.016497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:08.016497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:08.016497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:08.016497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:08.016497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:08.016497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:08.016497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:08.016497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:08.017629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:49:08.017629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:49:08.017629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:08.017629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:08.017629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 618/1242] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:49:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:08.022782 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ceefd-5aee-4791-93f0-e5e16d949454 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:08.136191 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:08.147211 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ceefd-5aee-4791-93f0-e5e16d949454 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:08.190373 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fec8ca-6bd4-4526-81dd-93a1b561ec75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:08.191041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fec8ca-6bd4-4526-81dd-93a1b561ec75 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:08.191327 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fec8ca-6bd4-4526-81dd-93a1b561ec75 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:08.191600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fec8ca-6bd4-4526-81dd-93a1b561ec75 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:08.192317 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fec8ca-6bd4-4526-81dd-93a1b561ec75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:08.192620 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fec8ca-6bd4-4526-81dd-93a1b561ec75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:08.193974 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fec8ca-6bd4-4526-81dd-93a1b561ec75 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:08.208465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 625/1243] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:08.214353 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9cc6dc-60ac-4684-9a9f-49ee07a16c4e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:08.304910 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:08.330444 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9cc6dc-60ac-4684-9a9f-49ee07a16c4e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:08.366981 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a2c7-7fa9-42bd-8706-7fb5933e122b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:08.367774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a2c7-7fa9-42bd-8706-7fb5933e122b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:08.368057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a2c7-7fa9-42bd-8706-7fb5933e122b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:08.368331 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a2c7-7fa9-42bd-8706-7fb5933e122b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:08.377369 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699a2c7-7fa9-42bd-8706-7fb5933e122b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:08.379076 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699a2c7-7fa9-42bd-8706-7fb5933e122b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:08.394070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-7699a2c7-7fa9-42bd-8706-7fb5933e122b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:49:08.394070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:08.394070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:08.394070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:08.394070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:08.394070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:08.394070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:08.394070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:08.394070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:08.394070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:08.394070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:08.394070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:08.394070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:49:08.394070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:49:08.394070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:49:08.395213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:49:08.395213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:08.395213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:08.395213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:08.395213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:08.395213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:08.395213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:08.395213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:08.395213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:08.395213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:08.395213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:08.395213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:08.395213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:08.395213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:08.395213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:08.395213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:08.395213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:08.396428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:49:08.396428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:49:08.396428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:49:08.396428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:49:08.396428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:49:08.396428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:49:08.396428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:49:08.396428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:08.396428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 619/1244] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:49:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:49:08.401103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fec8ca-6bd4-4526-81dd-93a1b561ec75 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:08.523312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fec8ca-6bd4-4526-81dd-93a1b561ec75 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:08.568898 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e710daf-af00-4c13-89de-61f112c0905a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:08.569663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e710daf-af00-4c13-89de-61f112c0905a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:08.569975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e710daf-af00-4c13-89de-61f112c0905a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:08.570363 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e710daf-af00-4c13-89de-61f112c0905a None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:08.571357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e710daf-af00-4c13-89de-61f112c0905a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:08.571746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e710daf-af00-4c13-89de-61f112c0905a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:08.573489 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e710daf-af00-4c13-89de-61f112c0905a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:08.588904 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 626/1245] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:49:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:08.596153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a2c7-7fa9-42bd-8706-7fb5933e122b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:08.735130 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a2c7-7fa9-42bd-8706-7fb5933e122b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:08.791646 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5eb98b-82a1-47a5-8254-26f13330ab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:08.792387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5eb98b-82a1-47a5-8254-26f13330ab87 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:08.792675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5eb98b-82a1-47a5-8254-26f13330ab87 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:08.792912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5eb98b-82a1-47a5-8254-26f13330ab87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:08.794052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5eb98b-82a1-47a5-8254-26f13330ab87 None admin] RBAC: Adding query filter params (role.id=c123034c11cc4d9ea6cab00e05bdcd53, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=880d0299c38c4eb897c669a512e599be) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:08.794469 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5eb98b-82a1-47a5-8254-26f13330ab87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:08.796634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5eb98b-82a1-47a5-8254-26f13330ab87 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:08.803422 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 620/1246] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:49:08 2022] GET /identity/v3/role_assignments?user.id=880d0299c38c4eb897c669a512e599be&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=c123034c11cc4d9ea6cab00e05bdcd53 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:49:11.186519 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016cb4c-7569-4ff6-a66b-884555ce385e None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:11.189778 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016cb4c-7569-4ff6-a66b-884555ce385e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:11.189778 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016cb4c-7569-4ff6-a66b-884555ce385e None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:11.189778 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 627/1247] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:49:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:11.195171 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b0d1e-eff2-4c1d-b9fb-54cf008cd172 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:11.195659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b0d1e-eff2-4c1d-b9fb-54cf008cd172 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:11.195837 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b0d1e-eff2-4c1d-b9fb-54cf008cd172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:11.201990 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:11.216058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-634b0d1e-eff2-4c1d-b9fb-54cf008cd172 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:11.217269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-634b0d1e-eff2-4c1d-b9fb-54cf008cd172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:11.312237 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 621/1248] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:49:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:49:11.343413 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016cb4c-7569-4ff6-a66b-884555ce385e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:11.371963 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:11.491182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016cb4c-7569-4ff6-a66b-884555ce385e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:11.552892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9154918-9ec4-4b65-9083-a0b80003297c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:11.553887 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9154918-9ec4-4b65-9083-a0b80003297c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:11.554170 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9154918-9ec4-4b65-9083-a0b80003297c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:11.554523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9154918-9ec4-4b65-9083-a0b80003297c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:11.568036 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9154918-9ec4-4b65-9083-a0b80003297c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:49:11.577730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-c9154918-9ec4-4b65-9083-a0b80003297c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:49:11.577730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:11.577730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:11.577730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:11.577730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:11.577730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:11.577730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:11.577730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:11.577730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:11.577730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:11.577730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:11.577730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:11.577730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:49:11.577730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:49:11.577730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:49:11.577730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:49:11.579097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:49:11.579097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:49:11.579097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:49:11.579097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:49:11.579097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:11.579097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:11.579097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:11.579097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:11.579097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:11.579097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:11.579097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:11.579097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:11.579097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:11.579097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:11.579097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:11.579097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:11.580893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:11.580893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:11.580893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:11.580893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:11.580893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:49:11.580893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:49:11.580893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:49:11.580893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:49:11.580893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:49:11.580893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:49:11.580893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:49:11.580893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:49:11.580893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 628/1249] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:49:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:11.585881 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b0d1e-eff2-4c1d-b9fb-54cf008cd172 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:11.722888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b0d1e-eff2-4c1d-b9fb-54cf008cd172 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:11.772391 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:11.782031 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18180d-1d06-48e3-99f7-8576a7a1b017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:11.783042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18180d-1d06-48e3-99f7-8576a7a1b017 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:11.783540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18180d-1d06-48e3-99f7-8576a7a1b017 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:11.783915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18180d-1d06-48e3-99f7-8576a7a1b017 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:11.785118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18180d-1d06-48e3-99f7-8576a7a1b017 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:11.785519 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18180d-1d06-48e3-99f7-8576a7a1b017 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:11.787302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18180d-1d06-48e3-99f7-8576a7a1b017 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:11.798902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 622/1250] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:49:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:11.806754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9154918-9ec4-4b65-9083-a0b80003297c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:11.911398 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:11.938974 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9154918-9ec4-4b65-9083-a0b80003297c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:11.984402 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0c713-890a-47cc-8245-f64cd080b576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:11.985187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0c713-890a-47cc-8245-f64cd080b576 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:11.985467 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0c713-890a-47cc-8245-f64cd080b576 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:11.985742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0c713-890a-47cc-8245-f64cd080b576 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:12.001552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0c713-890a-47cc-8245-f64cd080b576 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:12.003353 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0c713-890a-47cc-8245-f64cd080b576 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:12.026933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-19a0c713-890a-47cc-8245-f64cd080b576 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:12.026933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:12.026933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:12.026933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:12.026933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:12.026933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:12.026933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:12.026933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:12.026933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:12.026933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:12.026933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:12.026933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:12.026933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:49:12.026933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:49:12.026933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:49:12.028072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:49:12.028072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:12.028072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:12.028072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:12.028072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:12.028072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:12.028072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:12.028072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:12.028072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:12.028072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:12.028072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:12.028072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:12.028072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:12.028072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:12.028072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:12.028072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:12.028072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:12.029247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:49:12.029247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:49:12.029247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:12.029247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:49:12.030791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 629/1251] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:49:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:12.035006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18180d-1d06-48e3-99f7-8576a7a1b017 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:12.172861 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18180d-1d06-48e3-99f7-8576a7a1b017 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:12.218503 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd0e20-6bb8-4f1a-8672-e276a22f90ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:12.219261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd0e20-6bb8-4f1a-8672-e276a22f90ef None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:12.219506 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd0e20-6bb8-4f1a-8672-e276a22f90ef None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:12.219749 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd0e20-6bb8-4f1a-8672-e276a22f90ef None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:12.220547 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd0e20-6bb8-4f1a-8672-e276a22f90ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:12.220805 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd0e20-6bb8-4f1a-8672-e276a22f90ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:12.222167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd0e20-6bb8-4f1a-8672-e276a22f90ef None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:12.236503 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 623/1252] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:12.245606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0c713-890a-47cc-8245-f64cd080b576 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:12.383472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0c713-890a-47cc-8245-f64cd080b576 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:12.421446 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:12.431152 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aadab6-2601-470d-befe-ebae7d1c1e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:12.431970 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aadab6-2601-470d-befe-ebae7d1c1e7f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:12.432258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aadab6-2601-470d-befe-ebae7d1c1e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:12.432641 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aadab6-2601-470d-befe-ebae7d1c1e7f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:12.444733 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aadab6-2601-470d-befe-ebae7d1c1e7f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:12.447419 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aadab6-2601-470d-befe-ebae7d1c1e7f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:12.469430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-42aadab6-2601-470d-befe-ebae7d1c1e7f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:49:12.469430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:12.469430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:12.469430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:12.469430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:12.469430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:12.469430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:12.469430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:12.469430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:12.469430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:12.469430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:12.469430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:12.469430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:49:12.469430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:49:12.469430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:49:12.469430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:12.471000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:12.472273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:12.472273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:12.472273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:49:12.472273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:49:12.472273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:49:12.472273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:49:12.472273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:49:12.472273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:49:12.472273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:49:12.472273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:49:12.472273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 630/1253] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:49:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:12.477380 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd0e20-6bb8-4f1a-8672-e276a22f90ef None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:12.566874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:12.647574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd0e20-6bb8-4f1a-8672-e276a22f90ef None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:12.696211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f632c6-0a0e-4724-856a-b67dc3a6f5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:12.696936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f632c6-0a0e-4724-856a-b67dc3a6f5ee None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:12.697289 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f632c6-0a0e-4724-856a-b67dc3a6f5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:12.697504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f632c6-0a0e-4724-856a-b67dc3a6f5ee None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:12.698706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f632c6-0a0e-4724-856a-b67dc3a6f5ee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:12.699118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f632c6-0a0e-4724-856a-b67dc3a6f5ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:12.701688 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f632c6-0a0e-4724-856a-b67dc3a6f5ee None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:12.735985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 624/1254] 10.222.0.70 () {56 vars in 1143 bytes} [Thu Jun 16 18:49:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:12.742896 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aadab6-2601-470d-befe-ebae7d1c1e7f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:12.893735 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aadab6-2601-470d-befe-ebae7d1c1e7f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:12.941025 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad227f-f46f-4358-b954-901b28d96c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:12.941980 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad227f-f46f-4358-b954-901b28d96c8e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:12.942326 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad227f-f46f-4358-b954-901b28d96c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:12.943762 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad227f-f46f-4358-b954-901b28d96c8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:13.021882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:13.039904 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad227f-f46f-4358-b954-901b28d96c8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:13.041379 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad227f-f46f-4358-b954-901b28d96c8e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:13.175962 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 631/1255] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:49:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:49:13.183070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f632c6-0a0e-4724-856a-b67dc3a6f5ee None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:13.332533 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f632c6-0a0e-4724-856a-b67dc3a6f5ee None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:13.356508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:13.399006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafd71b-e8fd-4da5-8452-1b3a136049e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:13.400159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafd71b-e8fd-4da5-8452-1b3a136049e9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:13.400688 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafd71b-e8fd-4da5-8452-1b3a136049e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:13.401262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafd71b-e8fd-4da5-8452-1b3a136049e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:13.419220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafd71b-e8fd-4da5-8452-1b3a136049e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:13.421676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafd71b-e8fd-4da5-8452-1b3a136049e9 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:13.446455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-4dafd71b-e8fd-4da5-8452-1b3a136049e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:13.446455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:13.446455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:13.446455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:13.446455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:13.446455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:13.446455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:13.446455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:13.446455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:13.446455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:13.446455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:13.446455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:13.446455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:49:13.446455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:49:13.446455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:49:13.447504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:49:13.447504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:13.447504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:13.447504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:13.447504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:13.447504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:13.447504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:13.447504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:13.447504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:13.447504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:13.447504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:13.447504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:13.447504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:13.447504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:13.447504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:13.447504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:13.447504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:13.448822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:49:13.448822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:49:13.448822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:13.448822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:13.448822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 625/1256] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:13.454686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad227f-f46f-4358-b954-901b28d96c8e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:13.601491 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:13.619552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad227f-f46f-4358-b954-901b28d96c8e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:13.673689 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45369a40-4663-43fb-ac6b-bfb921ed7732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:13.674511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45369a40-4663-43fb-ac6b-bfb921ed7732 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:13.674939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45369a40-4663-43fb-ac6b-bfb921ed7732 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:13.675248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45369a40-4663-43fb-ac6b-bfb921ed7732 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:13.676107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45369a40-4663-43fb-ac6b-bfb921ed7732 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:13.676556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45369a40-4663-43fb-ac6b-bfb921ed7732 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:13.678371 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45369a40-4663-43fb-ac6b-bfb921ed7732 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:13.698424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 632/1257] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:13.701204 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafd71b-e8fd-4da5-8452-1b3a136049e9 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:13.855854 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafd71b-e8fd-4da5-8452-1b3a136049e9 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:13.906426 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81505fd-b173-4e5f-a1e8-94aff8130eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:13.907531 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81505fd-b173-4e5f-a1e8-94aff8130eed None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:13.907841 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81505fd-b173-4e5f-a1e8-94aff8130eed None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:13.908142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81505fd-b173-4e5f-a1e8-94aff8130eed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:13.917592 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81505fd-b173-4e5f-a1e8-94aff8130eed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:13.919274 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81505fd-b173-4e5f-a1e8-94aff8130eed None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:13.939678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e81505fd-b173-4e5f-a1e8-94aff8130eed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:49:13.939678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:13.939678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:13.939678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:13.939678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:13.939678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:13.939678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:13.939678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:13.939678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:13.939678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:13.939678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:13.939678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:13.939678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:49:13.939678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:49:13.939678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:49:13.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:49:13.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:13.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:13.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:13.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:13.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:13.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:13.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:13.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:13.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:13.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:13.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:13.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:13.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:13.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:13.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:13.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:13.946806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:49:13.946806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:49:13.946806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:49:13.946806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:49:13.946806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:49:13.946806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:49:13.946806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:49:13.946806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:13.946806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 626/1258] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:49:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:13.948048 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45369a40-4663-43fb-ac6b-bfb921ed7732 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:14.083006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45369a40-4663-43fb-ac6b-bfb921ed7732 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:14.122473 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa05f2c-8f48-405d-b55a-87d573518640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:14.123051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa05f2c-8f48-405d-b55a-87d573518640 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:14.123370 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa05f2c-8f48-405d-b55a-87d573518640 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:14.123657 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa05f2c-8f48-405d-b55a-87d573518640 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:14.124505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa05f2c-8f48-405d-b55a-87d573518640 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:14.124840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa05f2c-8f48-405d-b55a-87d573518640 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:14.126405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa05f2c-8f48-405d-b55a-87d573518640 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:14.143113 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 633/1259] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:49:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:14.158190 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81505fd-b173-4e5f-a1e8-94aff8130eed None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:14.188767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:14.286556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81505fd-b173-4e5f-a1e8-94aff8130eed None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:14.348355 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9c2d0-ed07-4a93-8fa6-457c5e0f07be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:14.349322 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9c2d0-ed07-4a93-8fa6-457c5e0f07be None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:14.349837 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9c2d0-ed07-4a93-8fa6-457c5e0f07be None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:14.350138 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9c2d0-ed07-4a93-8fa6-457c5e0f07be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:14.351497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9c2d0-ed07-4a93-8fa6-457c5e0f07be None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=880d0299c38c4eb897c669a512e599be) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:14.351929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9c2d0-ed07-4a93-8fa6-457c5e0f07be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:14.353643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9c2d0-ed07-4a93-8fa6-457c5e0f07be None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:14.361711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 627/1260] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:49:14 2022] GET /identity/v3/role_assignments?user.id=880d0299c38c4eb897c669a512e599be&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:16.599655 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2edce-ce58-4ff8-9441-72372273aa43 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:16.600248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2edce-ce58-4ff8-9441-72372273aa43 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:16.600248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2edce-ce58-4ff8-9441-72372273aa43 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:16.601579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 634/1261] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:16.607813 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373a11c-b1a1-480f-b192-410c5a46de08 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:16.608261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373a11c-b1a1-480f-b192-410c5a46de08 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:16.608373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373a11c-b1a1-480f-b192-410c5a46de08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:16.626255 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1373a11c-b1a1-480f-b192-410c5a46de08 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:16.627514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1373a11c-b1a1-480f-b192-410c5a46de08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:16.698046 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 628/1262] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:49:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:49:16.725362 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2edce-ce58-4ff8-9441-72372273aa43 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:16.738376 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:16.837204 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2edce-ce58-4ff8-9441-72372273aa43 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:16.873911 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f788ec5-993d-4d49-914f-3bfb3ef29174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:16.874754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f788ec5-993d-4d49-914f-3bfb3ef29174 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:16.874924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f788ec5-993d-4d49-914f-3bfb3ef29174 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:16.875184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f788ec5-993d-4d49-914f-3bfb3ef29174 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:16.885316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f788ec5-993d-4d49-914f-3bfb3ef29174 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:49:16.893276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-7f788ec5-993d-4d49-914f-3bfb3ef29174 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:49:16.893276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:16.893276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:16.893276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:16.893276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:16.893276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:16.893276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:16.893276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:16.893276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:16.893276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:16.893276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:16.893276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:16.893276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:49:16.893276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:49:16.893276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:49:16.893276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:49:16.894405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:49:16.894405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:49:16.894405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:49:16.894405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:49:16.894405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:16.894405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:16.894405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:16.894405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:16.894405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:16.894405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:16.894405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:16.894405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:16.894405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:16.894405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:16.894405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:16.894405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:16.895477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:16.895477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:16.895477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:16.895477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:16.895477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:49:16.895477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:49:16.895477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:49:16.895477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:49:16.895477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:49:16.895477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:49:16.895477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:49:16.895477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:49:16.895477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 635/1263] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:49:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:16.900717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373a11c-b1a1-480f-b192-410c5a46de08 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:16.921030 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:17.008751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373a11c-b1a1-480f-b192-410c5a46de08 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:17.045471 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c42d8-709e-492a-9e5d-33d9f1b26441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:17.046144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c42d8-709e-492a-9e5d-33d9f1b26441 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:17.046428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c42d8-709e-492a-9e5d-33d9f1b26441 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:17.046740 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c42d8-709e-492a-9e5d-33d9f1b26441 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:17.047486 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c42d8-709e-492a-9e5d-33d9f1b26441 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:17.047790 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c42d8-709e-492a-9e5d-33d9f1b26441 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:17.049158 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c42d8-709e-492a-9e5d-33d9f1b26441 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:17.058130 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 629/1264] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:49:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:17.064065 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f788ec5-993d-4d49-914f-3bfb3ef29174 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:17.141323 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:17.179074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f788ec5-993d-4d49-914f-3bfb3ef29174 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:17.214321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b7a1a-f382-4949-9412-f99dde913749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:17.216200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b7a1a-f382-4949-9412-f99dde913749 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:17.216484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b7a1a-f382-4949-9412-f99dde913749 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:17.216762 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b7a1a-f382-4949-9412-f99dde913749 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:17.227343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b7a1a-f382-4949-9412-f99dde913749 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:17.228781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b7a1a-f382-4949-9412-f99dde913749 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:17.247219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-730b7a1a-f382-4949-9412-f99dde913749 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:17.247219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:17.247219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:17.247219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:17.247219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:17.247219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:17.247219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:17.247219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:17.247219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:17.247219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:17.247219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:17.247219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:17.247219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:49:17.247219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:49:17.247219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:49:17.248331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:49:17.248331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:17.248331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:17.248331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:17.248331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:17.248331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:17.248331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:17.248331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:17.248331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:17.248331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:17.248331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:17.248331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:17.248331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:17.248331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:17.248331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:17.248331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:17.248331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:17.249250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:49:17.249250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:49:17.249250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:17.249250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:49:17.249250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 636/1265] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:49:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:17.254181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c42d8-709e-492a-9e5d-33d9f1b26441 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:17.344324 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:17.365933 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c42d8-709e-492a-9e5d-33d9f1b26441 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:17.401663 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c0a4d-c611-4e86-ac8a-077adb6241ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:17.402279 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c0a4d-c611-4e86-ac8a-077adb6241ed None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:17.403431 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c0a4d-c611-4e86-ac8a-077adb6241ed None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:17.403800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c0a4d-c611-4e86-ac8a-077adb6241ed None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:17.404526 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c0a4d-c611-4e86-ac8a-077adb6241ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:17.404856 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c0a4d-c611-4e86-ac8a-077adb6241ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:17.406280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c0a4d-c611-4e86-ac8a-077adb6241ed None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:17.419593 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 630/1266] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:17.426354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b7a1a-f382-4949-9412-f99dde913749 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:17.533972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b7a1a-f382-4949-9412-f99dde913749 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:17.554546 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:17.569041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba129a82-8f7f-46b1-9f5a-0ce1b80149ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:17.569849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba129a82-8f7f-46b1-9f5a-0ce1b80149ea None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:17.570132 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba129a82-8f7f-46b1-9f5a-0ce1b80149ea None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:17.570405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba129a82-8f7f-46b1-9f5a-0ce1b80149ea None admin] PATH_INFO: `/v3/users/placement` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:17.579157 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba129a82-8f7f-46b1-9f5a-0ce1b80149ea None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:17.580591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba129a82-8f7f-46b1-9f5a-0ce1b80149ea None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:17.595350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ba129a82-8f7f-46b1-9f5a-0ce1b80149ea None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:49:17.595350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:17.595350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:17.595350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:17.595350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:17.595350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:17.595350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:17.595350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:17.595350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:17.595350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:17.595350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:17.595350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:17.595350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:49:17.595350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:49:17.595350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:49:17.595350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:17.596635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:17.597599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:17.597599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:17.597599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:49:17.597599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:49:17.597599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:49:17.597599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:49:17.597599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:49:17.597599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:49:17.597599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:49:17.597599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:49:17.597599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 637/1267] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:49:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:17.602421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c0a4d-c611-4e86-ac8a-077adb6241ed None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:17.706455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c0a4d-c611-4e86-ac8a-077adb6241ed None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:17.750632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8468a8-07da-4764-bba3-102ffa005651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:17.751390 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8468a8-07da-4764-bba3-102ffa005651 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:17.751634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8468a8-07da-4764-bba3-102ffa005651 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:17.751905 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8468a8-07da-4764-bba3-102ffa005651 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:17.752757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8468a8-07da-4764-bba3-102ffa005651 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:17.753051 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8468a8-07da-4764-bba3-102ffa005651 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:17.754512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8468a8-07da-4764-bba3-102ffa005651 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:17.766913 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:17.780964 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 631/1268] 10.222.0.70 () {56 vars in 1143 bytes} [Thu Jun 16 18:49:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:17.786787 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba129a82-8f7f-46b1-9f5a-0ce1b80149ea None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:17.891547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba129a82-8f7f-46b1-9f5a-0ce1b80149ea None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:17.926257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc039d4-53dd-4622-ac4e-8195d7c2c8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:17.927078 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc039d4-53dd-4622-ac4e-8195d7c2c8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:17.927358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc039d4-53dd-4622-ac4e-8195d7c2c8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:17.927635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc039d4-53dd-4622-ac4e-8195d7c2c8c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:17.939438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc039d4-53dd-4622-ac4e-8195d7c2c8c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:17.940884 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc039d4-53dd-4622-ac4e-8195d7c2c8c3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:17.961284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ffc039d4-53dd-4622-ac4e-8195d7c2c8c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:17.961284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:17.961284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:17.961284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:17.961284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:17.961284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:17.961284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:17.961284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:17.961284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:17.961284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:17.961284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:17.961284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:17.961284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:49:17.961284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:49:17.961284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:49:17.962386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:49:17.962386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:17.962386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:17.962386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:17.962386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:17.962386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:17.962386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:17.962386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:17.962386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:17.962386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:17.962386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:17.962386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:17.962386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:17.962386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:17.962386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:17.962386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:17.962386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:17.963486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:49:17.963486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:49:17.963486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:17.963486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:49:17.963486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 638/1269] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:49:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:17.969329 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8468a8-07da-4764-bba3-102ffa005651 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:18.076443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8468a8-07da-4764-bba3-102ffa005651 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:18.114461 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546fbc4-c66c-46ac-b76b-c1bc92466b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:18.115304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546fbc4-c66c-46ac-b76b-c1bc92466b14 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:18.115584 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546fbc4-c66c-46ac-b76b-c1bc92466b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:18.115858 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546fbc4-c66c-46ac-b76b-c1bc92466b14 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:18.116582 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9546fbc4-c66c-46ac-b76b-c1bc92466b14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:18.116884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9546fbc4-c66c-46ac-b76b-c1bc92466b14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:18.118243 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9546fbc4-c66c-46ac-b76b-c1bc92466b14 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:18.131771 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 632/1270] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:18.138166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc039d4-53dd-4622-ac4e-8195d7c2c8c3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:18.172349 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:18.255391 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc039d4-53dd-4622-ac4e-8195d7c2c8c3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:18.295358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51417942-9093-4a6b-8d70-0e6c13a0580d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:18.296147 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51417942-9093-4a6b-8d70-0e6c13a0580d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:18.296427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51417942-9093-4a6b-8d70-0e6c13a0580d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:18.296698 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51417942-9093-4a6b-8d70-0e6c13a0580d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:18.303661 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51417942-9093-4a6b-8d70-0e6c13a0580d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:18.305432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51417942-9093-4a6b-8d70-0e6c13a0580d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:18.320434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-51417942-9093-4a6b-8d70-0e6c13a0580d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:49:18.320434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:18.320434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:18.320434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:18.320434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:18.320434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:18.320434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:18.320434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:18.320434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:18.320434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:18.320434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:18.320434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:18.320434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:49:18.320434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:49:18.320434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:49:18.321557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:49:18.321557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:18.321557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:18.321557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:18.321557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:18.321557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:18.321557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:18.321557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:18.321557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:18.321557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:18.321557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:18.321557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:18.321557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:18.321557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:18.321557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:18.321557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:18.321557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:18.322760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:49:18.322760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:49:18.322760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:49:18.322760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:49:18.322760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:49:18.322760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:49:18.322760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:49:18.322760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:49:18.322760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 639/1271] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:49:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:18.327533 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546fbc4-c66c-46ac-b76b-c1bc92466b14 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:18.389512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:18.432948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546fbc4-c66c-46ac-b76b-c1bc92466b14 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:18.468217 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48b11f-a2d6-41a2-bcaa-ba8410312400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:18.468217 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48b11f-a2d6-41a2-bcaa-ba8410312400 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:18.468522 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48b11f-a2d6-41a2-bcaa-ba8410312400 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:18.468716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48b11f-a2d6-41a2-bcaa-ba8410312400 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:18.469494 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48b11f-a2d6-41a2-bcaa-ba8410312400 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:18.469797 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48b11f-a2d6-41a2-bcaa-ba8410312400 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:18.471264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48b11f-a2d6-41a2-bcaa-ba8410312400 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:18.486299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 633/1272] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:49:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:18.492353 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51417942-9093-4a6b-8d70-0e6c13a0580d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:18.587412 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:18.598471 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51417942-9093-4a6b-8d70-0e6c13a0580d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:18.635081 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1600a7-8039-4e98-9bd7-200ca38e3f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:18.635974 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1600a7-8039-4e98-9bd7-200ca38e3f54 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:18.636256 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1600a7-8039-4e98-9bd7-200ca38e3f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:18.636530 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1600a7-8039-4e98-9bd7-200ca38e3f54 None admin] PATH_INFO: `/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/880d0299c38c4eb897c669a512e599be/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:18.638943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1600a7-8039-4e98-9bd7-200ca38e3f54 None admin] RBAC: Authorizing `identity:create_grant(project_id=34185125e75d4aad8ad5382ab22ee8a2, user_id=880d0299c38c4eb897c669a512e599be, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:18.640630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1600a7-8039-4e98-9bd7-200ca38e3f54 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:18.654369 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 640/1273] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Jun 16 18:49:18 2022] PUT /identity/v3/projects/34185125e75d4aad8ad5382ab22ee8a2/users/880d0299c38c4eb897c669a512e599be/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:49:20.877062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb69bc9-12d7-45a5-822a-8d9f4f2dc25a None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:20.877671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb69bc9-12d7-45a5-822a-8d9f4f2dc25a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:20.877671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb69bc9-12d7-45a5-822a-8d9f4f2dc25a None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:20.879016 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 634/1274] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:49:20.884374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81e5c8-9316-4fa9-8144-f7c193cd2935 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:20.884799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81e5c8-9316-4fa9-8144-f7c193cd2935 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:20.884914 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81e5c8-9316-4fa9-8144-f7c193cd2935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:20.901945 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-fc81e5c8-9316-4fa9-8144-f7c193cd2935 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:20.903885 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-fc81e5c8-9316-4fa9-8144-f7c193cd2935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:20.987463 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 641/1275] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:49:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:49:21.011282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb69bc9-12d7-45a5-822a-8d9f4f2dc25a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:21.129037 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb69bc9-12d7-45a5-822a-8d9f4f2dc25a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:21.157057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:21.181045 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1014d79-9d39-4665-b4ce-8ad617f1304b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:21.181842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1014d79-9d39-4665-b4ce-8ad617f1304b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:21.182127 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1014d79-9d39-4665-b4ce-8ad617f1304b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:21.182401 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1014d79-9d39-4665-b4ce-8ad617f1304b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:21.193761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1014d79-9d39-4665-b4ce-8ad617f1304b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:49:21.202523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-d1014d79-9d39-4665-b4ce-8ad617f1304b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:49:21.202523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:21.202523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:21.202523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:21.202523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:21.202523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:21.202523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:21.202523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:21.202523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:21.202523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:21.202523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:21.202523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:21.202523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:49:21.202523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:49:21.202523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:49:21.202523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:49:21.203667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:49:21.203667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:49:21.203667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:49:21.203667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:49:21.203667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:21.203667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:21.203667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:21.203667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:21.203667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:21.203667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:21.203667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:21.203667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:21.203667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:21.203667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:21.203667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:21.203667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:21.204808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:21.204808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:21.204808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:21.204808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:21.204808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:49:21.204808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:49:21.204808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:49:21.204808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:49:21.204808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:49:21.204808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:49:21.204808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:49:21.204808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:21.204808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 635/1276] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Jun 16 18:49:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:49:21.210060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81e5c8-9316-4fa9-8144-f7c193cd2935 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:21.305335 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:21.330964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81e5c8-9316-4fa9-8144-f7c193cd2935 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:21.381141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d231636-fb64-4508-9fa4-ad4f6b795883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:21.382066 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d231636-fb64-4508-9fa4-ad4f6b795883 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:21.382490 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d231636-fb64-4508-9fa4-ad4f6b795883 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:21.382935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d231636-fb64-4508-9fa4-ad4f6b795883 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:21.384102 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d231636-fb64-4508-9fa4-ad4f6b795883 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:21.384640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d231636-fb64-4508-9fa4-ad4f6b795883 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:21.387434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d231636-fb64-4508-9fa4-ad4f6b795883 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:21.401232 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 642/1277] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Jun 16 18:49:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:21.407447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1014d79-9d39-4665-b4ce-8ad617f1304b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:21.514928 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1014d79-9d39-4665-b4ce-8ad617f1304b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:21.552032 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b806d9-28a3-46cd-8636-0fdcd0c64d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:21.552817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b806d9-28a3-46cd-8636-0fdcd0c64d5a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:21.553097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b806d9-28a3-46cd-8636-0fdcd0c64d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:21.553367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b806d9-28a3-46cd-8636-0fdcd0c64d5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:21.565239 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b806d9-28a3-46cd-8636-0fdcd0c64d5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:21.566725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b806d9-28a3-46cd-8636-0fdcd0c64d5a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:21.584730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-86b806d9-28a3-46cd-8636-0fdcd0c64d5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:21.584730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:21.584730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:21.584730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:21.584730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:21.584730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:21.584730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:21.584730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:21.584730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:21.584730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:21.584730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:21.584730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:21.584730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:49:21.584730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:49:21.584730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:49:21.586546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:49:21.586546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:21.586546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:21.586546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:21.586546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:21.586546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:21.586546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:21.586546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:21.586546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:21.586546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:21.586546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:21.586546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:21.586546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:21.586546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:21.586546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:21.586546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:21.586546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:21.588269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:49:21.588269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:49:21.588269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:21.588269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:21.588269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 636/1278] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:21.592988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d231636-fb64-4508-9fa4-ad4f6b795883 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:21.697216 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d231636-fb64-4508-9fa4-ad4f6b795883 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:21.743387 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09f8a6-a351-411d-b7dc-2aa80d9e3a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:21.744197 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09f8a6-a351-411d-b7dc-2aa80d9e3a70 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:21.744449 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09f8a6-a351-411d-b7dc-2aa80d9e3a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:21.744798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09f8a6-a351-411d-b7dc-2aa80d9e3a70 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:21.745809 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09f8a6-a351-411d-b7dc-2aa80d9e3a70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:21.746886 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09f8a6-a351-411d-b7dc-2aa80d9e3a70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:21.748662 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09f8a6-a351-411d-b7dc-2aa80d9e3a70 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:21.763671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 643/1279] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:21.770331 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b806d9-28a3-46cd-8636-0fdcd0c64d5a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:21.807792 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:21.888028 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b806d9-28a3-46cd-8636-0fdcd0c64d5a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:21.923797 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c92831-18a0-4145-9f8d-57dd6701830d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:21.924647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c92831-18a0-4145-9f8d-57dd6701830d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:21.924932 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c92831-18a0-4145-9f8d-57dd6701830d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:21.925242 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c92831-18a0-4145-9f8d-57dd6701830d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:21.942622 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c92831-18a0-4145-9f8d-57dd6701830d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:21.944099 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c92831-18a0-4145-9f8d-57dd6701830d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:21.959465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b6c92831-18a0-4145-9f8d-57dd6701830d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:49:21.959465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:21.959465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:21.959465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:21.959465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:21.959465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:21.959465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:21.959465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:21.959465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:21.959465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:21.959465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:21.959465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:21.959465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:49:21.959465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:49:21.959465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:49:21.959465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:21.960966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:21.961956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:21.961956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:21.961956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:49:21.961956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:49:21.961956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:49:21.961956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:49:21.961956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:49:21.961956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:49:21.961956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:49:21.961956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:21.961956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 637/1280] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:49:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:21.966893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09f8a6-a351-411d-b7dc-2aa80d9e3a70 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:21.970822 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:22.082476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09f8a6-a351-411d-b7dc-2aa80d9e3a70 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:22.118053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3106b-57a1-4986-b713-f59bacd65496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:22.118755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3106b-57a1-4986-b713-f59bacd65496 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:22.119037 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3106b-57a1-4986-b713-f59bacd65496 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:22.119312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3106b-57a1-4986-b713-f59bacd65496 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:22.120156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3106b-57a1-4986-b713-f59bacd65496 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:22.120457 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3106b-57a1-4986-b713-f59bacd65496 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:22.122749 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3106b-57a1-4986-b713-f59bacd65496 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:22.143102 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 644/1281] 10.222.0.70 () {56 vars in 1143 bytes} [Thu Jun 16 18:49:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:22.149259 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c92831-18a0-4145-9f8d-57dd6701830d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:22.251747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:22.261682 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c92831-18a0-4145-9f8d-57dd6701830d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:22.298286 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b1c41-4ab2-47dd-b40f-a34cdf0183b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:22.298976 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b1c41-4ab2-47dd-b40f-a34cdf0183b2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:22.299261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b1c41-4ab2-47dd-b40f-a34cdf0183b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:22.299538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b1c41-4ab2-47dd-b40f-a34cdf0183b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:22.379505 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b1c41-4ab2-47dd-b40f-a34cdf0183b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:22.380956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b1c41-4ab2-47dd-b40f-a34cdf0183b2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:22.478843 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:22.497251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 638/1282] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Jun 16 18:49:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:49:22.503274 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3106b-57a1-4986-b713-f59bacd65496 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:22.570830 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:22.611762 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3106b-57a1-4986-b713-f59bacd65496 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:22.646644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab269815-10cd-436f-9f7a-adec489aaa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:22.647443 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab269815-10cd-436f-9f7a-adec489aaa7c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:22.647722 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab269815-10cd-436f-9f7a-adec489aaa7c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:22.647991 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab269815-10cd-436f-9f7a-adec489aaa7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:22.659500 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab269815-10cd-436f-9f7a-adec489aaa7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:22.660961 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab269815-10cd-436f-9f7a-adec489aaa7c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:22.679849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ab269815-10cd-436f-9f7a-adec489aaa7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:22.679849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:22.679849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:22.679849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:22.679849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:22.679849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:22.679849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:22.679849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:22.679849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:22.679849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:22.679849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:22.679849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:22.679849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:49:22.679849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:49:22.679849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:49:22.680984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:49:22.680984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:22.680984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:22.680984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:22.680984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:22.680984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:22.680984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:22.680984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:22.680984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:22.680984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:22.680984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:22.680984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:22.680984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:22.680984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:22.680984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:22.680984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:22.680984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:22.681992 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:49:22.681992 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:49:22.681992 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:49:22.681992 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:49:22.681992 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 645/1283] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:22.687067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b1c41-4ab2-47dd-b40f-a34cdf0183b2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:22.796783 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b1c41-4ab2-47dd-b40f-a34cdf0183b2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:22.832221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f2b95-d882-4421-835b-27913beb890f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:22.832887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f2b95-d882-4421-835b-27913beb890f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:22.833165 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f2b95-d882-4421-835b-27913beb890f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:22.833436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f2b95-d882-4421-835b-27913beb890f None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:22.834157 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f2b95-d882-4421-835b-27913beb890f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:22.834457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f2b95-d882-4421-835b-27913beb890f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:22.835861 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f2b95-d882-4421-835b-27913beb890f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:22.863704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 639/1284] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Jun 16 18:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:22.872176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab269815-10cd-436f-9f7a-adec489aaa7c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:22.983989 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab269815-10cd-436f-9f7a-adec489aaa7c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:22.999876 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:23.021237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152a0dc-b23f-4d20-9865-abd2b64c723a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:23.022031 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152a0dc-b23f-4d20-9865-abd2b64c723a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:23.022311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152a0dc-b23f-4d20-9865-abd2b64c723a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:23.022599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152a0dc-b23f-4d20-9865-abd2b64c723a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:23.029621 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152a0dc-b23f-4d20-9865-abd2b64c723a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:23.031110 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152a0dc-b23f-4d20-9865-abd2b64c723a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:23.045319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-6152a0dc-b23f-4d20-9865-abd2b64c723a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:49:23.045319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:23.045319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:23.045319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:23.045319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:23.045319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:23.045319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:23.045319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:23.045319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:23.045319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:23.045319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:23.045319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:23.045319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:49:23.045319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:49:23.045319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:49:23.046487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:49:23.046487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:23.046487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:23.046487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:23.046487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:23.046487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:23.046487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:23.046487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:23.046487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:23.046487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:23.046487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:23.046487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:23.046487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:23.046487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:23.046487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:23.046487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:23.046487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:23.048399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:49:23.048399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:49:23.048399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:49:23.048399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:49:23.048399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:49:23.048399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:49:23.048399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:49:23.048399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jun 16 18:49:23.048399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 646/1285] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Jun 16 18:49:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:23.052452 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f2b95-d882-4421-835b-27913beb890f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:23.110306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:23.165698 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f2b95-d882-4421-835b-27913beb890f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:23.209886 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dc886-968a-4b70-8f66-40fbbddacf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:23.210549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc886-968a-4b70-8f66-40fbbddacf64 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:23.210855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc886-968a-4b70-8f66-40fbbddacf64 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:23.211128 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc886-968a-4b70-8f66-40fbbddacf64 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:23.211919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dc886-968a-4b70-8f66-40fbbddacf64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:23.212226 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dc886-968a-4b70-8f66-40fbbddacf64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:23.213661 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dc886-968a-4b70-8f66-40fbbddacf64 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:23.227102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 640/1286] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Jun 16 18:49:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:23.233492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152a0dc-b23f-4d20-9865-abd2b64c723a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:23.346618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152a0dc-b23f-4d20-9865-abd2b64c723a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:23.383098 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875864b5-75d7-402c-8f04-af108e16bbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:23.383751 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875864b5-75d7-402c-8f04-af108e16bbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:23.384036 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875864b5-75d7-402c-8f04-af108e16bbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:23.384314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875864b5-75d7-402c-8f04-af108e16bbb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:23.385150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875864b5-75d7-402c-8f04-af108e16bbb8 None admin] RBAC: Adding query filter params (role.id=e0ab182b79ab44f592864af0ad8507c5, scope.project.id=34185125e75d4aad8ad5382ab22ee8a2, user.id=880d0299c38c4eb897c669a512e599be) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:23.385471 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875864b5-75d7-402c-8f04-af108e16bbb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:23.387173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875864b5-75d7-402c-8f04-af108e16bbb8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:23.394447 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 647/1287] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Jun 16 18:49:23 2022] GET /identity/v3/role_assignments?user.id=880d0299c38c4eb897c669a512e599be&scope.project.id=34185125e75d4aad8ad5382ab22ee8a2&role.id=e0ab182b79ab44f592864af0ad8507c5 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:25.626147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbdd5d-389e-4b8e-8c6b-957c9fe4fa09 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:25.626714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbdd5d-389e-4b8e-8c6b-957c9fe4fa09 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:25.626812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbdd5d-389e-4b8e-8c6b-957c9fe4fa09 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:25.628122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 641/1288] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:25.633458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670db37-11cd-4dc9-8156-2aeb21e6245e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:25.633920 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670db37-11cd-4dc9-8156-2aeb21e6245e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:25.634010 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670db37-11cd-4dc9-8156-2aeb21e6245e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:25.652949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f670db37-11cd-4dc9-8156-2aeb21e6245e None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:25.654189 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f670db37-11cd-4dc9-8156-2aeb21e6245e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:25.684536 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:25.728939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 648/1289] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:49:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:49:25.757446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbdd5d-389e-4b8e-8c6b-957c9fe4fa09 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:25.880759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbdd5d-389e-4b8e-8c6b-957c9fe4fa09 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:25.894249 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:25.918706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373d339-e511-44b7-b04e-c5af6a966b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:25.919526 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373d339-e511-44b7-b04e-c5af6a966b0a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:25.919810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373d339-e511-44b7-b04e-c5af6a966b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:25.920086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373d339-e511-44b7-b04e-c5af6a966b0a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:25.920740 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373d339-e511-44b7-b04e-c5af6a966b0a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:25.922102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373d339-e511-44b7-b04e-c5af6a966b0a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:25.934929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-d373d339-e511-44b7-b04e-c5af6a966b0a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:49:25.934929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:25.934929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:49:25.934929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:49:25.934929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:49:25.934929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:49:25.934929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:49:25.934929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:49:25.934929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:49:25.934929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:25.934929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:49:25.934929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:25.934929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:25.934929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:25.934929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:25.934929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:25.934929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:25.936338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:25.936338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:25.936338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:25.936338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:25.936338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:25.936338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:25.936338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:49:25.936338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:49:25.936338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:49:25.936338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:49:25.936338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:25.936338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:25.936338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:25.936338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:25.936338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:25.936338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:25.937192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:25.937192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:25.937192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:25.937192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:25.937192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:25.937192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:25.937192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:25.937192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:25.937192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:25.937192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:25.937192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:49:25.937192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:49:25.937192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:49:25.937192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:25.937192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 642/1290] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Jun 16 18:49:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:25.942699 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670db37-11cd-4dc9-8156-2aeb21e6245e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:26.050917 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670db37-11cd-4dc9-8156-2aeb21e6245e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:26.094481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a10d76-fd13-4c62-9c71-7d506651c961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:26.095172 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a10d76-fd13-4c62-9c71-7d506651c961 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:26.095451 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a10d76-fd13-4c62-9c71-7d506651c961 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:26.095721 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a10d76-fd13-4c62-9c71-7d506651c961 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:26.096423 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a10d76-fd13-4c62-9c71-7d506651c961 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:26.096727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a10d76-fd13-4c62-9c71-7d506651c961 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:26.098083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a10d76-fd13-4c62-9c71-7d506651c961 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:26.104141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 649/1291] 10.222.0.70 () {56 vars in 1116 bytes} [Thu Jun 16 18:49:25 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:26.110563 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373d339-e511-44b7-b04e-c5af6a966b0a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:26.222324 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373d339-e511-44b7-b04e-c5af6a966b0a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:26.263500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3dac55-e185-4dbc-8c29-6cd1decf54f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:26.264448 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dac55-e185-4dbc-8c29-6cd1decf54f9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:26.264731 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dac55-e185-4dbc-8c29-6cd1decf54f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:26.265004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dac55-e185-4dbc-8c29-6cd1decf54f9 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:26.265713 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3dac55-e185-4dbc-8c29-6cd1decf54f9 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:26.266018 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3dac55-e185-4dbc-8c29-6cd1decf54f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:26.268653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3dac55-e185-4dbc-8c29-6cd1decf54f9 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:26.272201 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 643/1292] 10.222.0.70 () {56 vars in 1116 bytes} [Thu Jun 16 18:49:26 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:28.482604 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daeb38-8976-443a-bd6b-a9de0278aca5 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:28.483133 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daeb38-8976-443a-bd6b-a9de0278aca5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:28.483259 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daeb38-8976-443a-bd6b-a9de0278aca5 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:28.484534 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 650/1293] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:28.490030 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1405039-bf6d-43dc-b4b2-9a143de7be45 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:28.490449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1405039-bf6d-43dc-b4b2-9a143de7be45 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:28.490596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1405039-bf6d-43dc-b4b2-9a143de7be45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:28.507986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a1405039-bf6d-43dc-b4b2-9a143de7be45 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:28.509151 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a1405039-bf6d-43dc-b4b2-9a143de7be45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:28.539237 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:28.578026 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 644/1294] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:49:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:49:28.601404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03daeb38-8976-443a-bd6b-a9de0278aca5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:28.632953 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:28.710377 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03daeb38-8976-443a-bd6b-a9de0278aca5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:28.757917 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc82c2-5738-41f8-b7eb-46f6a41b5bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:28.758511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc82c2-5738-41f8-b7eb-46f6a41b5bea None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:28.758922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc82c2-5738-41f8-b7eb-46f6a41b5bea None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:28.759206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc82c2-5738-41f8-b7eb-46f6a41b5bea None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:28.760251 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc82c2-5738-41f8-b7eb-46f6a41b5bea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:28.762480 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc82c2-5738-41f8-b7eb-46f6a41b5bea None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:28.779633 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 651/1295] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Jun 16 18:49:28 2022] POST /identity/v3/services => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:49:31.023940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74e34a-12ec-4e26-aa3b-80a7ad3f6569 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:31.024469 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74e34a-12ec-4e26-aa3b-80a7ad3f6569 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:31.024469 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74e34a-12ec-4e26-aa3b-80a7ad3f6569 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:31.025851 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 645/1296] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:31.031891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f90eb2-6496-4afe-87ce-87340f0fd237 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:31.032338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f90eb2-6496-4afe-87ce-87340f0fd237 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:31.032508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f90eb2-6496-4afe-87ce-87340f0fd237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:31.050886 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-04f90eb2-6496-4afe-87ce-87340f0fd237 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:31.052281 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-04f90eb2-6496-4afe-87ce-87340f0fd237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:31.130898 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 652/1297] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:49:31 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:49:31.156517 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74e34a-12ec-4e26-aa3b-80a7ad3f6569 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:31.268774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74e34a-12ec-4e26-aa3b-80a7ad3f6569 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:31.304143 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397df0b9-25e2-471b-9a15-12468697a847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:31.304944 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397df0b9-25e2-471b-9a15-12468697a847 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:31.305230 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397df0b9-25e2-471b-9a15-12468697a847 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:31.305507 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397df0b9-25e2-471b-9a15-12468697a847 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:31.306159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397df0b9-25e2-471b-9a15-12468697a847 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:31.307577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397df0b9-25e2-471b-9a15-12468697a847 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:31.319776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-397df0b9-25e2-471b-9a15-12468697a847 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:49:31.319776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:31.319776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:49:31.319776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:49:31.319776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:49:31.319776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:49:31.319776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:49:31.319776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:49:31.319776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:49:31.319776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:31.319776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:49:31.319776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:31.319776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:31.319776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:31.319776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:31.319776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:31.319776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:31.321075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:31.321075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:31.321075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:31.321075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:31.321075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:31.321075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:31.321075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:49:31.321075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:49:31.321075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:49:31.321075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:49:31.321075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:31.321075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:31.321075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:31.321075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:31.321075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:31.321075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:31.322235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:31.322235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:31.322235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:31.322235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:31.322235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:31.322235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:31.322235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:31.322235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:31.322235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:31.322235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:31.322235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:49:31.322235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:49:31.322235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:49:31.322235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:31.322235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 646/1298] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Jun 16 18:49:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:31.328056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f90eb2-6496-4afe-87ce-87340f0fd237 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:31.348533 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:31.433368 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f90eb2-6496-4afe-87ce-87340f0fd237 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:31.469837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160f5b5-e401-4e9e-8477-fcfd00bef2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:31.470497 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160f5b5-e401-4e9e-8477-fcfd00bef2ce None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:31.470845 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160f5b5-e401-4e9e-8477-fcfd00bef2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:31.471116 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160f5b5-e401-4e9e-8477-fcfd00bef2ce None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:31.471817 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160f5b5-e401-4e9e-8477-fcfd00bef2ce None admin] RBAC: Adding query filter params (name=placement) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:31.472122 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160f5b5-e401-4e9e-8477-fcfd00bef2ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:31.473540 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160f5b5-e401-4e9e-8477-fcfd00bef2ce None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:31.479330 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 653/1299] 10.222.0.70 () {56 vars in 1116 bytes} [Thu Jun 16 18:49:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:31.486146 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397df0b9-25e2-471b-9a15-12468697a847 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:31.519434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:31.591692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397df0b9-25e2-471b-9a15-12468697a847 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:31.625866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2602d48-87fc-4c5a-8061-419e50fadd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:31.626528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2602d48-87fc-4c5a-8061-419e50fadd91 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:31.626845 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2602d48-87fc-4c5a-8061-419e50fadd91 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:31.627122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2602d48-87fc-4c5a-8061-419e50fadd91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:31.627907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2602d48-87fc-4c5a-8061-419e50fadd91 None admin] RBAC: Adding query filter params (interface=public, service_id=dd95236fd646476c9aff654f040648df, region_id=RegionOne) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:31.628216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2602d48-87fc-4c5a-8061-419e50fadd91 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:31.629590 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2602d48-87fc-4c5a-8061-419e50fadd91 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:31.634748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 647/1300] 10.222.0.70 () {56 vars in 1251 bytes} [Thu Jun 16 18:49:31 2022] GET /identity/v3/endpoints?service_id=dd95236fd646476c9aff654f040648df&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:31.641445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160f5b5-e401-4e9e-8477-fcfd00bef2ce None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:31.733252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:31.751878 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160f5b5-e401-4e9e-8477-fcfd00bef2ce None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:31.787323 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75f643-c57e-445a-a45c-1367d761e68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:31.787897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75f643-c57e-445a-a45c-1367d761e68b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:31.788182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75f643-c57e-445a-a45c-1367d761e68b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:31.788459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75f643-c57e-445a-a45c-1367d761e68b None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:31.789137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d75f643-c57e-445a-a45c-1367d761e68b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:31.790499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d75f643-c57e-445a-a45c-1367d761e68b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:31.798237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 654/1301] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Jun 16 18:49:31 2022] GET /identity/v3/services => generated 2191 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:49:34.016252 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc266e7-717f-4c4e-b4c7-958d87bc26d2 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:34.016823 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc266e7-717f-4c4e-b4c7-958d87bc26d2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:34.016823 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc266e7-717f-4c4e-b4c7-958d87bc26d2 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:34.018188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 648/1302] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:34.023528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b26263-fc4e-4ef5-bf73-61cb1fc304f4 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:34.023950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b26263-fc4e-4ef5-bf73-61cb1fc304f4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:34.024086 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b26263-fc4e-4ef5-bf73-61cb1fc304f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:34.047267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-46b26263-fc4e-4ef5-bf73-61cb1fc304f4 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:34.048466 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-46b26263-fc4e-4ef5-bf73-61cb1fc304f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:34.127902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 655/1303] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:49:34 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:49:34.154937 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc266e7-717f-4c4e-b4c7-958d87bc26d2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:34.272245 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:34.279656 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc266e7-717f-4c4e-b4c7-958d87bc26d2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:34.324883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c4f70-8282-4228-b4f8-3c7bc3d7ed9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:34.325701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c4f70-8282-4228-b4f8-3c7bc3d7ed9c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:34.325987 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c4f70-8282-4228-b4f8-3c7bc3d7ed9c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:34.326298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c4f70-8282-4228-b4f8-3c7bc3d7ed9c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:34.327004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c4f70-8282-4228-b4f8-3c7bc3d7ed9c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:34.328390 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c4f70-8282-4228-b4f8-3c7bc3d7ed9c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:34.341233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e35c4f70-8282-4228-b4f8-3c7bc3d7ed9c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:49:34.341233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:34.341233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:49:34.341233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:49:34.341233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:49:34.341233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:49:34.341233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:49:34.341233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:49:34.341233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:49:34.341233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:34.341233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:49:34.341233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:34.341233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:49:34.341233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:49:34.341233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:49:34.341233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:49:34.341233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:49:34.342485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:49:34.342485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:49:34.342485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:49:34.342485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:49:34.342485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:49:34.342485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:49:34.342485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:49:34.342485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:49:34.342485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:49:34.342485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:49:34.342485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:49:34.342485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:49:34.342485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:49:34.342485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:49:34.342485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:49:34.342485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:49:34.344817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:49:34.344817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jun 16 18:49:34.344817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:49:34.344817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:49:34.344817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:49:34.344817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:49:34.344817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:49:34.344817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:49:34.344817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:49:34.344817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:49:34.344817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:49:34.344817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:49:34.344817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:49:34.344817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jun 16 18:49:34.344817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 649/1304] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Jun 16 18:49:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:49:34.349606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b26263-fc4e-4ef5-bf73-61cb1fc304f4 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:34.446132 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:34.475289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b26263-fc4e-4ef5-bf73-61cb1fc304f4 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:34.514261 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e671cca-71a9-48fd-862f-bfb1e2248990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:34.515182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e671cca-71a9-48fd-862f-bfb1e2248990 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:34.515459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e671cca-71a9-48fd-862f-bfb1e2248990 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:34.515731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e671cca-71a9-48fd-862f-bfb1e2248990 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:34.516445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e671cca-71a9-48fd-862f-bfb1e2248990 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:49:34.516749 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e671cca-71a9-48fd-862f-bfb1e2248990 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:34.518996 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e671cca-71a9-48fd-862f-bfb1e2248990 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:34.525989 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 656/1305] 10.222.0.70 () {56 vars in 1116 bytes} [Thu Jun 16 18:49:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:49:34.533210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c4f70-8282-4228-b4f8-3c7bc3d7ed9c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:34.657769 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c4f70-8282-4228-b4f8-3c7bc3d7ed9c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:34.694814 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f60922-d5fd-41fb-b450-c292783aa0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:34.695484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f60922-d5fd-41fb-b450-c292783aa0a2 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:34.695774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f60922-d5fd-41fb-b450-c292783aa0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:34.696054 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f60922-d5fd-41fb-b450-c292783aa0a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:34.696808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f60922-d5fd-41fb-b450-c292783aa0a2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:34.698187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f60922-d5fd-41fb-b450-c292783aa0a2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:34.721381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 650/1306] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Jun 16 18:49:34 2022] POST /identity/v3/endpoints => generated 331 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:49:44.523217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=30 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:44.716183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:50:42.852991 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97d72c-60b3-44a9-ad84-a36abd03851f None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:42.855595 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97d72c-60b3-44a9-ad84-a36abd03851f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:42.856238 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97d72c-60b3-44a9-ad84-a36abd03851f None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:42.859094 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 657/1307] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:50:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:50:42.868788 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167445da-2456-4f1a-9fec-468fc42baa0e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:42.869818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167445da-2456-4f1a-9fec-468fc42baa0e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:42.870271 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167445da-2456-4f1a-9fec-468fc42baa0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:43.087197 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-167445da-2456-4f1a-9fec-468fc42baa0e None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:50:43.088758 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-167445da-2456-4f1a-9fec-468fc42baa0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:43.423170 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 651/1308] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:50:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 557 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:50:43.474990 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fee98-b25e-4524-ab13-7f3188149a43 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:43.475150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fee98-b25e-4524-ab13-7f3188149a43 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:43.475541 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fee98-b25e-4524-ab13-7f3188149a43 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:43.476867 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 658/1309] 10.222.0.70 () {54 vars in 964 bytes} [Thu Jun 16 18:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:50:43.484666 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d05eda-c93a-402a-bbb6-572f5c9fa16a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:43.485219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d05eda-c93a-402a-bbb6-572f5c9fa16a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:43.485439 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d05eda-c93a-402a-bbb6-572f5c9fa16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:43.665686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-84d05eda-c93a-402a-bbb6-572f5c9fa16a None None] MFA Rules not processed for user `02a83f6e7aa448d6984a78914c1748cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:50:43.666885 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-84d05eda-c93a-402a-bbb6-572f5c9fa16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:43.736356 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:50:43.816647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 652/1310] 10.222.0.70 () {58 vars in 1061 bytes} [Thu Jun 16 18:50:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:50:43.830400 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fee98-b25e-4524-ab13-7f3188149a43 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:50:43.832581 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-636fee98-b25e-4524-ab13-7f3188149a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:43.994563 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fee98-b25e-4524-ab13-7f3188149a43 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:50:44.095745 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ebdaf2-7d18-43fa-aca5-606d1cad9a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:50:44.096355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebdaf2-7d18-43fa-aca5-606d1cad9a9b service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:44.096638 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebdaf2-7d18-43fa-aca5-606d1cad9a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:44.096915 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebdaf2-7d18-43fa-aca5-606d1cad9a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:44.099055 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b2ebdaf2-7d18-43fa-aca5-606d1cad9a9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:44.164121 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:50:44.231455 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ebdaf2-7d18-43fa-aca5-606d1cad9a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:50:44.233099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ebdaf2-7d18-43fa-aca5-606d1cad9a9b service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:50:44.254043 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 659/1311] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:50:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:50:47.289233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d1cba-9d3e-4c47-a2de-c05224622209 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:47.290378 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d1cba-9d3e-4c47-a2de-c05224622209 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:47.290378 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d1cba-9d3e-4c47-a2de-c05224622209 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:47.295040 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 653/1312] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:50:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:50:47.301496 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d921f23-5b1b-4091-b8f3-2db38e8b355e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:47.302164 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d921f23-5b1b-4091-b8f3-2db38e8b355e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:47.302714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d921f23-5b1b-4091-b8f3-2db38e8b355e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:47.410335 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1d921f23-5b1b-4091-b8f3-2db38e8b355e None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:50:47.411619 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1d921f23-5b1b-4091-b8f3-2db38e8b355e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:47.534939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 660/1313] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:50:47.575542 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d1cba-9d3e-4c47-a2de-c05224622209 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:50:47.577343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d16d1cba-9d3e-4c47-a2de-c05224622209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:47.704164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d1cba-9d3e-4c47-a2de-c05224622209 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:50:47.800758 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:50:47.804590 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db844d02-2498-4080-ba81-52704cc3184a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:50:47.805278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db844d02-2498-4080-ba81-52704cc3184a service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:47.806135 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db844d02-2498-4080-ba81-52704cc3184a service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:47.807416 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db844d02-2498-4080-ba81-52704cc3184a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:47.809898 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-db844d02-2498-4080-ba81-52704cc3184a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:47.986158 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db844d02-2498-4080-ba81-52704cc3184a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:50:47.988383 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db844d02-2498-4080-ba81-52704cc3184a service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:50:48.009926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 654/1314] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:50:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:50:50.540131 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db92735-45d0-4a77-aa29-2f1dde57e54d None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:50.540778 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db92735-45d0-4a77-aa29-2f1dde57e54d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:50.540879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db92735-45d0-4a77-aa29-2f1dde57e54d None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:50.543394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 661/1315] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:50:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:50:50.549560 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46463c-4bdd-4418-8be3-823de6ec322e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:50.550034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46463c-4bdd-4418-8be3-823de6ec322e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:50.550155 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46463c-4bdd-4418-8be3-823de6ec322e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:50.625521 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ac46463c-4bdd-4418-8be3-823de6ec322e None None] MFA Rules not processed for user `c02531851c6c49c9abab77229f96d67f`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:50:50.626811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ac46463c-4bdd-4418-8be3-823de6ec322e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:50.683936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:50:50.741065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 655/1316] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:50:50 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:50:50.784295 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db92735-45d0-4a77-aa29-2f1dde57e54d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:50:50.785830 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0db92735-45d0-4a77-aa29-2f1dde57e54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:50.798281 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:50:50.901338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db92735-45d0-4a77-aa29-2f1dde57e54d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:50:50.984523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079b4fa-6111-4d86-8aca-36d114bc521c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:50:50.985443 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079b4fa-6111-4d86-8aca-36d114bc521c service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:50.985876 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079b4fa-6111-4d86-8aca-36d114bc521c service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:50.986329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079b4fa-6111-4d86-8aca-36d114bc521c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:50.988567 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7079b4fa-6111-4d86-8aca-36d114bc521c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:51.058660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:50:51.109728 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079b4fa-6111-4d86-8aca-36d114bc521c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:50:51.111215 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079b4fa-6111-4d86-8aca-36d114bc521c service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:50:51.130505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 662/1317] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:50:50 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:50:54.002701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d53c3d-13fd-41a2-ade7-357222f008a0 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:54.003239 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d53c3d-13fd-41a2-ade7-357222f008a0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:54.003239 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d53c3d-13fd-41a2-ade7-357222f008a0 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:54.004628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 656/1318] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:50:54.010471 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b556da-d123-423c-97cc-7d6cc9bc324b None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:54.010971 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b556da-d123-423c-97cc-7d6cc9bc324b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:54.011084 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b556da-d123-423c-97cc-7d6cc9bc324b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:54.077696 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-23b556da-d123-423c-97cc-7d6cc9bc324b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:50:54.078868 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-23b556da-d123-423c-97cc-7d6cc9bc324b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:54.178245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 663/1319] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:50:54.230031 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f24cc-2439-4ed9-b595-aae20ad06c29 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:54.230322 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f24cc-2439-4ed9-b595-aae20ad06c29 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:54.230637 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f24cc-2439-4ed9-b595-aae20ad06c29 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:54.232020 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 657/1320] 10.222.0.70 () {54 vars in 964 bytes} [Thu Jun 16 18:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:50:54.238005 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4873a00-c9b3-472b-9f08-d0483c5c34d0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:54.238292 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4873a00-c9b3-472b-9f08-d0483c5c34d0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:54.238579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4873a00-c9b3-472b-9f08-d0483c5c34d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:54.260845 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:50:54.319898 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b4873a00-c9b3-472b-9f08-d0483c5c34d0 None None] MFA Rules not processed for user `02a83f6e7aa448d6984a78914c1748cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:50:54.321144 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b4873a00-c9b3-472b-9f08-d0483c5c34d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:54.397797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 664/1321] 10.222.0.70 () {58 vars in 1061 bytes} [Thu Jun 16 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:50:54.407869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f24cc-2439-4ed9-b595-aae20ad06c29 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:50:54.409386 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3a0f24cc-2439-4ed9-b595-aae20ad06c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:54.512632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f24cc-2439-4ed9-b595-aae20ad06c29 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:50:54.585785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf5777-3517-4edb-999d-59b9c32a9364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:50:54.586432 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf5777-3517-4edb-999d-59b9c32a9364 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:54.586785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf5777-3517-4edb-999d-59b9c32a9364 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:54.587227 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf5777-3517-4edb-999d-59b9c32a9364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:54.588986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-96cf5777-3517-4edb-999d-59b9c32a9364 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:54.596355 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:50:54.700103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cf5777-3517-4edb-999d-59b9c32a9364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:50:54.701535 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cf5777-3517-4edb-999d-59b9c32a9364 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:50:54.721413 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 658/1322] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:50:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:50:58.064584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abf66a-8c81-48b7-8537-9224d9033126 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:58.065145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abf66a-8c81-48b7-8537-9224d9033126 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:58.065145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abf66a-8c81-48b7-8537-9224d9033126 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:58.067188 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 665/1323] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:50:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:50:58.075689 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2c609-a775-40e5-8007-d4281d5c0300 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:58.076235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2c609-a775-40e5-8007-d4281d5c0300 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:58.076235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2c609-a775-40e5-8007-d4281d5c0300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:58.165383 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ada2c609-a775-40e5-8007-d4281d5c0300 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:50:58.169968 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ada2c609-a775-40e5-8007-d4281d5c0300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:58.235201 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:50:58.281982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 659/1324] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:50:58.323447 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abf66a-8c81-48b7-8537-9224d9033126 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:50:58.326159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-78abf66a-8c81-48b7-8537-9224d9033126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:58.440337 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abf66a-8c81-48b7-8537-9224d9033126 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:50:58.453740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:50:58.524961 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea4264-c016-42ff-8640-082bebe2a558 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:50:58.525764 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea4264-c016-42ff-8640-082bebe2a558 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:58.526051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea4264-c016-42ff-8640-082bebe2a558 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:58.526332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea4264-c016-42ff-8640-082bebe2a558 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:58.528042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b2ea4264-c016-42ff-8640-082bebe2a558 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:58.659690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea4264-c016-42ff-8640-082bebe2a558 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:50:58.662225 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea4264-c016-42ff-8640-082bebe2a558 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:50:58.685435 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 666/1325] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:50:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:51:01.003904 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908dee4-be48-4a45-875f-53a103c26e66 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:01.004437 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908dee4-be48-4a45-875f-53a103c26e66 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:01.004557 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908dee4-be48-4a45-875f-53a103c26e66 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:01.005838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 660/1326] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:01.012459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a85f2-7946-4219-9dd6-58ec18ad14bc None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:01.012940 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a85f2-7946-4219-9dd6-58ec18ad14bc None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:01.013035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a85f2-7946-4219-9dd6-58ec18ad14bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:01.063585 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:51:01.084424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c98a85f2-7946-4219-9dd6-58ec18ad14bc None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:01.085638 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c98a85f2-7946-4219-9dd6-58ec18ad14bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:01.189089 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 667/1327] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:51:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:01.227110 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908dee4-be48-4a45-875f-53a103c26e66 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:51:01.228947 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2908dee4-be48-4a45-875f-53a103c26e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:01.337065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908dee4-be48-4a45-875f-53a103c26e66 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:51:01.423087 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64134b92-1d75-47fa-8972-24939408e436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:51:01.423757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64134b92-1d75-47fa-8972-24939408e436 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:01.424043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64134b92-1d75-47fa-8972-24939408e436 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:01.424322 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64134b92-1d75-47fa-8972-24939408e436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:01.425995 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-64134b92-1d75-47fa-8972-24939408e436 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:01.443950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:51:01.539802 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64134b92-1d75-47fa-8972-24939408e436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:51:01.541837 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64134b92-1d75-47fa-8972-24939408e436 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:51:01.560544 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 661/1328] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:51:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:51:03.832325 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafc8d4-ac19-4e6e-8606-09fe6301971d None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:03.832832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafc8d4-ac19-4e6e-8606-09fe6301971d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:03.832930 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafc8d4-ac19-4e6e-8606-09fe6301971d None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:03.834407 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 668/1329] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:03.841133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8a278-f335-423b-b9f7-c84ef38646f1 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:03.841865 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8a278-f335-423b-b9f7-c84ef38646f1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:03.842022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8a278-f335-423b-b9f7-c84ef38646f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:03.927904 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e6e8a278-f335-423b-b9f7-c84ef38646f1 None None] MFA Rules not processed for user `c02531851c6c49c9abab77229f96d67f`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:03.929144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e6e8a278-f335-423b-b9f7-c84ef38646f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:04.013921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:51:04.064963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 662/1330] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:51:03 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:04.100622 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafc8d4-ac19-4e6e-8606-09fe6301971d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:51:04.102391 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-daafc8d4-ac19-4e6e-8606-09fe6301971d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:04.237560 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafc8d4-ac19-4e6e-8606-09fe6301971d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:51:04.260295 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:51:04.322259 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26dcb5-fed4-4f8c-b199-f3f552e796ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:51:04.323063 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26dcb5-fed4-4f8c-b199-f3f552e796ee service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:04.323401 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26dcb5-fed4-4f8c-b199-f3f552e796ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:04.323740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26dcb5-fed4-4f8c-b199-f3f552e796ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:04.325913 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1f26dcb5-fed4-4f8c-b199-f3f552e796ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:04.456935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26dcb5-fed4-4f8c-b199-f3f552e796ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:51:04.458407 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26dcb5-fed4-4f8c-b199-f3f552e796ee service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:51:04.480138 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 669/1331] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:51:04 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:51:06.998331 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077cd15-ca95-4e72-a5b5-bbbb336c1bcd None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:06.998891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077cd15-ca95-4e72-a5b5-bbbb336c1bcd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:06.998891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077cd15-ca95-4e72-a5b5-bbbb336c1bcd None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:07.000303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 663/1332] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:07.005390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aaac0a-4fca-4134-95ae-bbabd83ca9f4 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:07.005811 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aaac0a-4fca-4134-95ae-bbabd83ca9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:07.005952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aaac0a-4fca-4134-95ae-bbabd83ca9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:07.069323 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:51:07.072588 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b3aaac0a-4fca-4134-95ae-bbabd83ca9f4 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:07.074417 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b3aaac0a-4fca-4134-95ae-bbabd83ca9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:07.172149 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 670/1333] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:51:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:07.208503 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077cd15-ca95-4e72-a5b5-bbbb336c1bcd None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:51:07.210023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c077cd15-ca95-4e72-a5b5-bbbb336c1bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:07.312437 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077cd15-ca95-4e72-a5b5-bbbb336c1bcd None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:51:07.387392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1abd41-a0c5-46bc-8d46-6a3cfb26c031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:51:07.388042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1abd41-a0c5-46bc-8d46-6a3cfb26c031 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:07.388326 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1abd41-a0c5-46bc-8d46-6a3cfb26c031 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:07.388602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1abd41-a0c5-46bc-8d46-6a3cfb26c031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:07.390223 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4a1abd41-a0c5-46bc-8d46-6a3cfb26c031 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:07.418907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:51:07.501249 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1abd41-a0c5-46bc-8d46-6a3cfb26c031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:51:07.502608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1abd41-a0c5-46bc-8d46-6a3cfb26c031 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:51:07.520970 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 664/1334] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:51:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:51:09.760823 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2cba8-37df-4667-887f-1a8232a0ba05 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:09.761431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2cba8-37df-4667-887f-1a8232a0ba05 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:09.761431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2cba8-37df-4667-887f-1a8232a0ba05 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:09.762779 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 671/1335] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:51:09.768364 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba78a5-e3b8-45e3-b3e1-031ef0c0a909 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:09.768799 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba78a5-e3b8-45e3-b3e1-031ef0c0a909 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:09.768921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba78a5-e3b8-45e3-b3e1-031ef0c0a909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:09.835364 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-98ba78a5-e3b8-45e3-b3e1-031ef0c0a909 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:09.836551 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-98ba78a5-e3b8-45e3-b3e1-031ef0c0a909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:09.911654 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:51:09.935262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 665/1336] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:09.975140 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2cba8-37df-4667-887f-1a8232a0ba05 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:51:09.976681 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-76c2cba8-37df-4667-887f-1a8232a0ba05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:10.081295 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2cba8-37df-4667-887f-1a8232a0ba05 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:51:10.110353 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:51:10.158459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0af9b3-2ab4-43ee-91cb-0a17f72d3a13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:51:10.159137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0af9b3-2ab4-43ee-91cb-0a17f72d3a13 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:10.159419 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0af9b3-2ab4-43ee-91cb-0a17f72d3a13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:10.159694 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0af9b3-2ab4-43ee-91cb-0a17f72d3a13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:10.161317 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4e0af9b3-2ab4-43ee-91cb-0a17f72d3a13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:10.270300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0af9b3-2ab4-43ee-91cb-0a17f72d3a13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:51:10.271783 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0af9b3-2ab4-43ee-91cb-0a17f72d3a13 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:51:10.289470 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 672/1337] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:51:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:51:12.917406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dab1cd-4289-47b6-95a5-edab734de732 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:12.917972 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dab1cd-4289-47b6-95a5-edab734de732 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:12.917972 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dab1cd-4289-47b6-95a5-edab734de732 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:12.919378 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 666/1338] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:51:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:12.924554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d2862-b91b-4b0c-9913-85f3bef46ff3 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:12.924974 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d2862-b91b-4b0c-9913-85f3bef46ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:12.925127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d2862-b91b-4b0c-9913-85f3bef46ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:12.990842 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-031d2862-b91b-4b0c-9913-85f3bef46ff3 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:12.992071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-031d2862-b91b-4b0c-9913-85f3bef46ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:13.001008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:51:13.091230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 673/1339] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:51:12 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:13.127273 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dab1cd-4289-47b6-95a5-edab734de732 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:51:13.130357 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f8dab1cd-4289-47b6-95a5-edab734de732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:13.238586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dab1cd-4289-47b6-95a5-edab734de732 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:51:13.252332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df13f36-ee1f-4d88-85b6-f43dad143e79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:51:13.252974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df13f36-ee1f-4d88-85b6-f43dad143e79 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:13.253252 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df13f36-ee1f-4d88-85b6-f43dad143e79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:13.253524 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df13f36-ee1f-4d88-85b6-f43dad143e79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:13.255176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0df13f36-ee1f-4d88-85b6-f43dad143e79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:13.355751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:51:13.358025 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df13f36-ee1f-4d88-85b6-f43dad143e79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:51:13.359778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df13f36-ee1f-4d88-85b6-f43dad143e79 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:51:13.379593 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 667/1340] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:51:13 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:51:17.692801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfd4bd-6886-4488-91c0-dc8a51b93ecf None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:17.693344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfd4bd-6886-4488-91c0-dc8a51b93ecf None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:17.693344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfd4bd-6886-4488-91c0-dc8a51b93ecf None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:17.694739 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 674/1341] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:17.700529 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6908578-1e87-4bdf-ac65-78908368f491 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:17.701011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6908578-1e87-4bdf-ac65-78908368f491 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:17.701122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6908578-1e87-4bdf-ac65-78908368f491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:17.750481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f6908578-1e87-4bdf-ac65-78908368f491 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:17.751855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f6908578-1e87-4bdf-ac65-78908368f491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:17.837366 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 668/1342] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:17.875000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfd4bd-6886-4488-91c0-dc8a51b93ecf None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:51:17.877191 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-35dfd4bd-6886-4488-91c0-dc8a51b93ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:17.902698 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfd4bd-6886-4488-91c0-dc8a51b93ecf None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:51:17.906230 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0104f-62b2-43b4-b17e-03076cf0a6fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:51:17.906907 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0104f-62b2-43b4-b17e-03076cf0a6fd service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:17.907193 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0104f-62b2-43b4-b17e-03076cf0a6fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:17.907466 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0104f-62b2-43b4-b17e-03076cf0a6fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:17.917923 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0104f-62b2-43b4-b17e-03076cf0a6fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:51:17.919399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0104f-62b2-43b4-b17e-03076cf0a6fd service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:51:17.929393 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 675/1343] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:51:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:51:20.674482 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738f31c-f79f-46bd-adb6-9a4f8f841f9d None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:20.675051 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738f31c-f79f-46bd-adb6-9a4f8f841f9d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:20.675051 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738f31c-f79f-46bd-adb6-9a4f8f841f9d None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:20.676479 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 669/1344] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:20.684760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889fa02-64c7-4dd1-aae1-33fe73f7f3ed None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:20.685265 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889fa02-64c7-4dd1-aae1-33fe73f7f3ed None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:20.685361 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889fa02-64c7-4dd1-aae1-33fe73f7f3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:20.713600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-8889fa02-64c7-4dd1-aae1-33fe73f7f3ed None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:20.716697 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8889fa02-64c7-4dd1-aae1-33fe73f7f3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:20.725206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 676/1345] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:51:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:20.778191 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738f31c-f79f-46bd-adb6-9a4f8f841f9d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:51:20.791604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738f31c-f79f-46bd-adb6-9a4f8f841f9d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:51:20.796429 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe877c-a983-409c-9c29-3fdd82923bc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:51:20.797059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe877c-a983-409c-9c29-3fdd82923bc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:20.797337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe877c-a983-409c-9c29-3fdd82923bc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:20.797604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe877c-a983-409c-9c29-3fdd82923bc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:20.809820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe877c-a983-409c-9c29-3fdd82923bc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:51:20.811291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe877c-a983-409c-9c29-3fdd82923bc9 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:51:20.824685 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 670/1346] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:51:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:51:25.220533 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d4056-5ac2-402d-ad40-8e82e2c7bce5 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:25.221080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d4056-5ac2-402d-ad40-8e82e2c7bce5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:25.221227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d4056-5ac2-402d-ad40-8e82e2c7bce5 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:25.222680 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 677/1347] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:25.228642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a626d63-c898-43da-9af1-1c78a694a198 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:25.229686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a626d63-c898-43da-9af1-1c78a694a198 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:25.230039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a626d63-c898-43da-9af1-1c78a694a198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:25.249403 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3a626d63-c898-43da-9af1-1c78a694a198 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:25.250655 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3a626d63-c898-43da-9af1-1c78a694a198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:25.259714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 671/1348] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:25.299029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d4056-5ac2-402d-ad40-8e82e2c7bce5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:51:25.310695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d4056-5ac2-402d-ad40-8e82e2c7bce5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:51:25.314758 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661575e-65cc-4aab-ab7d-0ff15e0bc74d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:51:25.315126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661575e-65cc-4aab-ab7d-0ff15e0bc74d service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:25.315405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661575e-65cc-4aab-ab7d-0ff15e0bc74d service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:25.315666 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661575e-65cc-4aab-ab7d-0ff15e0bc74d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:25.325768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661575e-65cc-4aab-ab7d-0ff15e0bc74d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:51:25.327229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661575e-65cc-4aab-ab7d-0ff15e0bc74d service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:51:25.337137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 678/1349] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:51:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:51:30.593403 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bb226-9579-4066-adf9-c3ed9339930b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:30.594363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bb226-9579-4066-adf9-c3ed9339930b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:30.594886 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bb226-9579-4066-adf9-c3ed9339930b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:30.596502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 672/1350] 10.222.0.70 () {54 vars in 921 bytes} [Thu Jun 16 18:51:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:30.621409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eff9c0-cea3-4967-89b8-32b86881d681 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:30.622319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eff9c0-cea3-4967-89b8-32b86881d681 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:30.622835 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eff9c0-cea3-4967-89b8-32b86881d681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:30.700657 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-12eff9c0-cea3-4967-89b8-32b86881d681 None None] MFA Rules not processed for user `880d0299c38c4eb897c669a512e599be`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:30.702398 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-12eff9c0-cea3-4967-89b8-32b86881d681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:30.745833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:51:30.800411 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 679/1351] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Jun 16 18:51:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:51:35.106153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32eb63-7ce3-4242-8bbb-d76610186bf0 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:35.106748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32eb63-7ce3-4242-8bbb-d76610186bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:35.106923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32eb63-7ce3-4242-8bbb-d76610186bf0 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:35.108613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 673/1352] 10.222.0.70 () {54 vars in 921 bytes} [Thu Jun 16 18:51:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:35.113873 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7849a-a95c-438d-ba27-a47fd57d9f26 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:35.114308 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7849a-a95c-438d-ba27-a47fd57d9f26 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:35.114435 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7849a-a95c-438d-ba27-a47fd57d9f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:35.135107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5cc7849a-a95c-438d-ba27-a47fd57d9f26 None None] MFA Rules not processed for user `880d0299c38c4eb897c669a512e599be`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:35.136505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5cc7849a-a95c-438d-ba27-a47fd57d9f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:35.145913 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 680/1353] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Jun 16 18:51:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:35.242451 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=60 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:51:36.808354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4efd0-343e-48e7-9fd3-2e1f8cef2279 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:36.809102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4efd0-343e-48e7-9fd3-2e1f8cef2279 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:36.809405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4efd0-343e-48e7-9fd3-2e1f8cef2279 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:36.811549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 674/1354] 10.222.0.70 () {54 vars in 921 bytes} [Thu Jun 16 18:51:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:36.816801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc8705-e2cb-47da-b11c-f739d53efb06 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:36.817253 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc8705-e2cb-47da-b11c-f739d53efb06 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:36.817352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc8705-e2cb-47da-b11c-f739d53efb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:36.839003 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-16fc8705-e2cb-47da-b11c-f739d53efb06 None None] MFA Rules not processed for user `880d0299c38c4eb897c669a512e599be`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:36.840247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-16fc8705-e2cb-47da-b11c-f739d53efb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:36.849268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 681/1355] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Jun 16 18:51:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:51:40.703089 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50f730-7c74-4beb-a6fb-9a8bfa27549f None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:40.704006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50f730-7c74-4beb-a6fb-9a8bfa27549f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:40.704459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50f730-7c74-4beb-a6fb-9a8bfa27549f None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:40.706448 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 675/1356] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:51:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:40.712467 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54660-2ace-4ebe-8629-2bc499fb43f2 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:40.713387 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54660-2ace-4ebe-8629-2bc499fb43f2 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:40.713387 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54660-2ace-4ebe-8629-2bc499fb43f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:40.735307 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b3f54660-2ace-4ebe-8629-2bc499fb43f2 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:40.736847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b3f54660-2ace-4ebe-8629-2bc499fb43f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:40.747233 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 682/1357] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:51:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:51:40.801664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5b23c-0664-45a5-b2d4-226525b2bc70 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:40.801959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5b23c-0664-45a5-b2d4-226525b2bc70 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:40.802244 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5b23c-0664-45a5-b2d4-226525b2bc70 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:40.804364 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 676/1358] 10.222.0.70 () {54 vars in 961 bytes} [Thu Jun 16 18:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:40.815866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdd53e-0024-40e0-b213-5da947b3e031 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:40.818954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdd53e-0024-40e0-b213-5da947b3e031 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:40.823375 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdd53e-0024-40e0-b213-5da947b3e031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:40.914972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f3fdd53e-0024-40e0-b213-5da947b3e031 None None] MFA Rules not processed for user `8342da838fe4462b9de2817ad3ac4ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:40.917047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f3fdd53e-0024-40e0-b213-5da947b3e031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:41.031689 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 683/1359] 10.222.0.70 () {58 vars in 1058 bytes} [Thu Jun 16 18:51:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:41.042983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5b23c-0664-45a5-b2d4-226525b2bc70 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:51:41.063069 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5b23c-0664-45a5-b2d4-226525b2bc70 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:51:41.069813 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd4f32-e2b4-4130-ba25-43f3a118806c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:51:41.070786 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd4f32-e2b4-4130-ba25-43f3a118806c service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:41.071208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd4f32-e2b4-4130-ba25-43f3a118806c service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:41.071608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd4f32-e2b4-4130-ba25-43f3a118806c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:41.082034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd4f32-e2b4-4130-ba25-43f3a118806c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:51:41.083648 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd4f32-e2b4-4130-ba25-43f3a118806c service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:51:41.104371 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 677/1360] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:51:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:51:43.545760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b90023-8ecb-42d1-ac9b-d520e53ebe72 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:43.546326 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b90023-8ecb-42d1-ac9b-d520e53ebe72 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:43.546326 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b90023-8ecb-42d1-ac9b-d520e53ebe72 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:43.548572 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 684/1361] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:43.553935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084d0c8-fd70-4de6-9827-23d96fa646d2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:43.554432 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084d0c8-fd70-4de6-9827-23d96fa646d2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:43.554539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084d0c8-fd70-4de6-9827-23d96fa646d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:43.572807 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a084d0c8-fd70-4de6-9827-23d96fa646d2 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:43.574482 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a084d0c8-fd70-4de6-9827-23d96fa646d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:43.582689 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 678/1362] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:51:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:51:43.630751 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b90023-8ecb-42d1-ac9b-d520e53ebe72 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:51:43.648778 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b90023-8ecb-42d1-ac9b-d520e53ebe72 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:51:43.656800 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb7093-060f-4ca3-8abb-a775f8c477f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:51:43.657423 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb7093-060f-4ca3-8abb-a775f8c477f9 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:43.657710 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb7093-060f-4ca3-8abb-a775f8c477f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:43.657983 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb7093-060f-4ca3-8abb-a775f8c477f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:43.668615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb7093-060f-4ca3-8abb-a775f8c477f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:51:43.670561 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb7093-060f-4ca3-8abb-a775f8c477f9 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:51:43.683371 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 685/1363] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:51:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:51:46.300668 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bc96c-b3c1-4c0d-be07-b7ad03f396d3 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:46.301209 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bc96c-b3c1-4c0d-be07-b7ad03f396d3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:46.301209 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bc96c-b3c1-4c0d-be07-b7ad03f396d3 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:46.302633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 679/1364] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:46.309935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c38290-e946-41f8-8500-c80f332dcb82 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:46.310662 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c38290-e946-41f8-8500-c80f332dcb82 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:46.310794 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c38290-e946-41f8-8500-c80f332dcb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:46.333999 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d9c38290-e946-41f8-8500-c80f332dcb82 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:46.335342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d9c38290-e946-41f8-8500-c80f332dcb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:46.344098 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 686/1365] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:46.399067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f392c-dffb-4aa5-9784-47c0419acbea None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:46.399356 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f392c-dffb-4aa5-9784-47c0419acbea None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:46.399633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f392c-dffb-4aa5-9784-47c0419acbea None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:46.401751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 680/1366] 10.222.0.70 () {54 vars in 961 bytes} [Thu Jun 16 18:51:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:46.408231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2802402-e352-45e5-b9d0-404d2e61d4cf None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:46.408753 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2802402-e352-45e5-b9d0-404d2e61d4cf None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:46.409252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2802402-e352-45e5-b9d0-404d2e61d4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:46.432950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-a2802402-e352-45e5-b9d0-404d2e61d4cf None None] MFA Rules not processed for user `8342da838fe4462b9de2817ad3ac4ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:46.433593 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a2802402-e352-45e5-b9d0-404d2e61d4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:46.445458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 687/1367] 10.222.0.70 () {58 vars in 1058 bytes} [Thu Jun 16 18:51:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:46.454538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f392c-dffb-4aa5-9784-47c0419acbea None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:51:46.472288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f392c-dffb-4aa5-9784-47c0419acbea None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:51:46.478227 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582e7d8-61b2-446a-90bd-1ac53cd9e2ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:51:46.480011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582e7d8-61b2-446a-90bd-1ac53cd9e2ef service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:46.480295 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582e7d8-61b2-446a-90bd-1ac53cd9e2ef service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:46.480571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582e7d8-61b2-446a-90bd-1ac53cd9e2ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:46.489986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582e7d8-61b2-446a-90bd-1ac53cd9e2ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:51:46.491611 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582e7d8-61b2-446a-90bd-1ac53cd9e2ef service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:51:46.502816 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 681/1368] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:51:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:51:48.748566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653dc98-a822-495d-b216-b6d503784eca None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:48.749125 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653dc98-a822-495d-b216-b6d503784eca None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:48.749125 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653dc98-a822-495d-b216-b6d503784eca None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:48.750484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 688/1369] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:48.755905 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d8764-201d-4fb2-b4a6-0590330bc595 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:48.756296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d8764-201d-4fb2-b4a6-0590330bc595 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:48.756467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d8764-201d-4fb2-b4a6-0590330bc595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:48.774926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-bf4d8764-201d-4fb2-b4a6-0590330bc595 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:48.777351 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bf4d8764-201d-4fb2-b4a6-0590330bc595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:48.785582 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 682/1370] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:48.830315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653dc98-a822-495d-b216-b6d503784eca None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:51:48.846043 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653dc98-a822-495d-b216-b6d503784eca None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:51:48.851724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd857c-f155-49de-87e0-2882e0811927 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:51:48.852462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd857c-f155-49de-87e0-2882e0811927 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:48.852748 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd857c-f155-49de-87e0-2882e0811927 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:48.853023 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd857c-f155-49de-87e0-2882e0811927 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:48.863449 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd857c-f155-49de-87e0-2882e0811927 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:51:48.865054 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd857c-f155-49de-87e0-2882e0811927 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:51:48.875581 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 689/1371] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:51:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:51:49.169781 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b586a-648d-4e0a-a7bc-507b886729e4 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:49.170323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b586a-648d-4e0a-a7bc-507b886729e4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:49.170799 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b586a-648d-4e0a-a7bc-507b886729e4 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:49.172352 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 683/1372] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:51:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:51:49.177773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc21b41-7dd6-4d1d-9e56-9ee11ee9b637 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:49.178241 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc21b41-7dd6-4d1d-9e56-9ee11ee9b637 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:49.178741 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc21b41-7dd6-4d1d-9e56-9ee11ee9b637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:49.198046 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-adc21b41-7dd6-4d1d-9e56-9ee11ee9b637 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:49.199694 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-adc21b41-7dd6-4d1d-9e56-9ee11ee9b637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:49.208141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 690/1373] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:51:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:49.521818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b20a5a-613c-40b8-ae09-4bbab6507318 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:49.521818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b20a5a-613c-40b8-ae09-4bbab6507318 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:49.521818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b20a5a-613c-40b8-ae09-4bbab6507318 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:49.524210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 684/1374] 10.222.0.70 () {54 vars in 963 bytes} [Thu Jun 16 18:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:49.526159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4c281-9a66-485e-a720-7a7001572863 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:49.526693 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4c281-9a66-485e-a720-7a7001572863 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:49.527571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4c281-9a66-485e-a720-7a7001572863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:49.554885 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:51:49.584568 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-aad4c281-9a66-485e-a720-7a7001572863 None None] MFA Rules not processed for user `a47de73e3f3b4979bff83140bf287275`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:49.586168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-aad4c281-9a66-485e-a720-7a7001572863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:49.654237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 691/1375] 10.222.0.70 () {58 vars in 1060 bytes} [Thu Jun 16 18:51:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:49.662386 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b20a5a-613c-40b8-ae09-4bbab6507318 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:51:49.674810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b20a5a-613c-40b8-ae09-4bbab6507318 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:51:49.679207 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c105e10-f3ab-4b92-9f94-dd7911ee4603 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:51:49.680030 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c105e10-f3ab-4b92-9f94-dd7911ee4603 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:49.680458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c105e10-f3ab-4b92-9f94-dd7911ee4603 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:49.680886 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c105e10-f3ab-4b92-9f94-dd7911ee4603 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:49.692024 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c105e10-f3ab-4b92-9f94-dd7911ee4603 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:51:49.693629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c105e10-f3ab-4b92-9f94-dd7911ee4603 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:51:49.705829 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 685/1376] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:51:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:51:51.092494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ca21d-844f-45d8-a9e1-741fa0df6f99 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:51.093048 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ca21d-844f-45d8-a9e1-741fa0df6f99 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:51.093048 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ca21d-844f-45d8-a9e1-741fa0df6f99 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:51.094403 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 692/1377] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:51.099676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2d744-de10-4bea-b6ea-b9136b261406 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:51.100111 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2d744-de10-4bea-b6ea-b9136b261406 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:51.100245 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2d744-de10-4bea-b6ea-b9136b261406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:51.124011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b4e2d744-de10-4bea-b6ea-b9136b261406 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:51.125218 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b4e2d744-de10-4bea-b6ea-b9136b261406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:51.133151 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 686/1378] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:51.179762 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ca21d-844f-45d8-a9e1-741fa0df6f99 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:51:51.194003 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ca21d-844f-45d8-a9e1-741fa0df6f99 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:51:51.201731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3ba1e-7f34-40c6-b739-101e6b004c7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:51:51.202328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3ba1e-7f34-40c6-b739-101e6b004c7d service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:51.202642 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3ba1e-7f34-40c6-b739-101e6b004c7d service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:51.202921 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3ba1e-7f34-40c6-b739-101e6b004c7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:51.212563 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3ba1e-7f34-40c6-b739-101e6b004c7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:51:51.214748 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3ba1e-7f34-40c6-b739-101e6b004c7d service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:51:51.225883 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 693/1379] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:51:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:51:53.607589 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799d44e-f693-49f0-8894-cb944f45869c None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:53.608126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799d44e-f693-49f0-8894-cb944f45869c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:53.608126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799d44e-f693-49f0-8894-cb944f45869c None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:53.609389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 687/1380] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:53.615243 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b5242-0c03-4996-88be-463ab145bf53 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:53.616067 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b5242-0c03-4996-88be-463ab145bf53 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:53.616503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b5242-0c03-4996-88be-463ab145bf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:53.640366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-092b5242-0c03-4996-88be-463ab145bf53 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:53.641590 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-092b5242-0c03-4996-88be-463ab145bf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:53.650498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 694/1381] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:51:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:53.698250 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799d44e-f693-49f0-8894-cb944f45869c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:51:53.714858 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799d44e-f693-49f0-8894-cb944f45869c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:51:53.721891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb210278-bd1b-4ae0-9fb8-5157909b9145 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:51:53.722710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb210278-bd1b-4ae0-9fb8-5157909b9145 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:53.723031 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb210278-bd1b-4ae0-9fb8-5157909b9145 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:53.723323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb210278-bd1b-4ae0-9fb8-5157909b9145 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:53.733106 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb210278-bd1b-4ae0-9fb8-5157909b9145 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:51:53.734758 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb210278-bd1b-4ae0-9fb8-5157909b9145 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:51:53.745871 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 688/1382] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:51:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:51:56.050431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e0bbe-8c5d-4506-bcd3-80608bf79fe4 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:56.051141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e0bbe-8c5d-4506-bcd3-80608bf79fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:56.051141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e0bbe-8c5d-4506-bcd3-80608bf79fe4 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:56.052744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 695/1383] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:56.059678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7a54e-3a8d-4801-b8c1-3156bf2f524c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:56.060398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7a54e-3a8d-4801-b8c1-3156bf2f524c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:56.060398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7a54e-3a8d-4801-b8c1-3156bf2f524c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:56.081895 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-03a7a54e-3a8d-4801-b8c1-3156bf2f524c None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:56.083159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-03a7a54e-3a8d-4801-b8c1-3156bf2f524c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:56.092326 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 689/1384] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:51:56.137014 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e0bbe-8c5d-4506-bcd3-80608bf79fe4 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:51:56.154339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e0bbe-8c5d-4506-bcd3-80608bf79fe4 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:51:56.160182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf910a-3483-4371-91fb-05844b550d60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:51:56.161000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf910a-3483-4371-91fb-05844b550d60 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:56.161292 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf910a-3483-4371-91fb-05844b550d60 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:56.161570 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf910a-3483-4371-91fb-05844b550d60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:56.171603 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf910a-3483-4371-91fb-05844b550d60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:51:56.173205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf910a-3483-4371-91fb-05844b550d60 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:51:56.190821 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 696/1385] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:51:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:51:58.652266 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b301a9-c720-4b82-a9ec-8d51ea721c8a None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:58.652838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b301a9-c720-4b82-a9ec-8d51ea721c8a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:58.652838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b301a9-c720-4b82-a9ec-8d51ea721c8a None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:58.654193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 690/1386] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:51:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:51:58.659628 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c167c6a-3109-4add-9153-e1e4e2e6705d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:58.660128 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c167c6a-3109-4add-9153-e1e4e2e6705d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:58.660240 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c167c6a-3109-4add-9153-e1e4e2e6705d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:58.679130 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-8c167c6a-3109-4add-9153-e1e4e2e6705d None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:51:58.680363 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8c167c6a-3109-4add-9153-e1e4e2e6705d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:51:58.689678 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 697/1387] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:51:58.734927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b301a9-c720-4b82-a9ec-8d51ea721c8a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:51:58.750113 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b301a9-c720-4b82-a9ec-8d51ea721c8a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:51:58.755748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b1253-16a3-46ff-a24a-67aa88631044 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:51:58.756396 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b1253-16a3-46ff-a24a-67aa88631044 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:51:58.756681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b1253-16a3-46ff-a24a-67aa88631044 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:51:58.756968 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b1253-16a3-46ff-a24a-67aa88631044 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:51:58.769739 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b1253-16a3-46ff-a24a-67aa88631044 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:51:58.771553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b1253-16a3-46ff-a24a-67aa88631044 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:51:58.781996 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 691/1388] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:51:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:01.101291 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f159d1-d897-4bff-933e-e808e87f5142 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:01.102024 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f159d1-d897-4bff-933e-e808e87f5142 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:01.102024 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f159d1-d897-4bff-933e-e808e87f5142 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:01.103329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 698/1389] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:01.111860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aaad03-c89c-48a9-a098-287f893553c9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:01.112496 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aaad03-c89c-48a9-a098-287f893553c9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:01.112496 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aaad03-c89c-48a9-a098-287f893553c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:01.130410 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c7aaad03-c89c-48a9-a098-287f893553c9 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:01.131578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c7aaad03-c89c-48a9-a098-287f893553c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:01.139575 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 692/1390] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:52:01.189358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f159d1-d897-4bff-933e-e808e87f5142 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:01.210871 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f159d1-d897-4bff-933e-e808e87f5142 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:01.218271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a77660-36d1-4537-a065-e7db376ef780 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:01.218950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a77660-36d1-4537-a065-e7db376ef780 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:01.219232 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a77660-36d1-4537-a065-e7db376ef780 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:01.219505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a77660-36d1-4537-a065-e7db376ef780 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:01.228423 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a77660-36d1-4537-a065-e7db376ef780 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:01.230741 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a77660-36d1-4537-a065-e7db376ef780 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:01.240694 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 699/1391] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:03.458416 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc1647-76f2-401f-b6aa-7a87735ea669 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:03.459276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc1647-76f2-401f-b6aa-7a87735ea669 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:03.459276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc1647-76f2-401f-b6aa-7a87735ea669 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:03.460368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 693/1392] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:03.465619 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937db47-175f-4f35-8a3b-e256d8a90bf0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:03.466201 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937db47-175f-4f35-8a3b-e256d8a90bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:03.466201 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937db47-175f-4f35-8a3b-e256d8a90bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:03.484730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-a937db47-175f-4f35-8a3b-e256d8a90bf0 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:03.485968 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a937db47-175f-4f35-8a3b-e256d8a90bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:03.493915 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 700/1393] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:03.540106 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc1647-76f2-401f-b6aa-7a87735ea669 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:03.559706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc1647-76f2-401f-b6aa-7a87735ea669 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:03.566176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff52c5-4c93-41e8-8485-bd0b602f3f64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:03.567099 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff52c5-4c93-41e8-8485-bd0b602f3f64 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:03.567483 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff52c5-4c93-41e8-8485-bd0b602f3f64 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:03.567856 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff52c5-4c93-41e8-8485-bd0b602f3f64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:03.578847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff52c5-4c93-41e8-8485-bd0b602f3f64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:03.580774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff52c5-4c93-41e8-8485-bd0b602f3f64 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:03.594183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 694/1394] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:05.792465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c83cd-44ea-46af-8961-455daa718050 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:05.793136 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c83cd-44ea-46af-8961-455daa718050 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:05.793136 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c83cd-44ea-46af-8961-455daa718050 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:05.794462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 701/1395] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:52:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:05.799770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042fd62-ffa1-4614-a03f-6b358fd970d8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:05.800221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042fd62-ffa1-4614-a03f-6b358fd970d8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:05.800319 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042fd62-ffa1-4614-a03f-6b358fd970d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:05.806485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=52 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:52:05.820198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e042fd62-ffa1-4614-a03f-6b358fd970d8 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:05.821434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e042fd62-ffa1-4614-a03f-6b358fd970d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:05.829581 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 695/1396] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:05.874955 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c83cd-44ea-46af-8961-455daa718050 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:05.890776 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c83cd-44ea-46af-8961-455daa718050 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:05.896715 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da56646-12e5-434b-b7cc-824f2d87a381 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:05.897269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39007162-0d86-41f4-b77d-481b596855f6 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:05.897408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da56646-12e5-434b-b7cc-824f2d87a381 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:05.897555 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39007162-0d86-41f4-b77d-481b596855f6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:05.897661 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da56646-12e5-434b-b7cc-824f2d87a381 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:05.897832 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39007162-0d86-41f4-b77d-481b596855f6 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:05.897936 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da56646-12e5-434b-b7cc-824f2d87a381 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:05.899234 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 696/1397] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:52:05.905314 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcdbe0-9709-4ac3-94dc-5c963b80788c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:05.905655 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcdbe0-9709-4ac3-94dc-5c963b80788c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:05.905969 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcdbe0-9709-4ac3-94dc-5c963b80788c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:05.907855 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da56646-12e5-434b-b7cc-824f2d87a381 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:05.909458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da56646-12e5-434b-b7cc-824f2d87a381 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:05.919955 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 702/1398] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:05.924273 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f3fcdbe0-9709-4ac3-94dc-5c963b80788c None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:05.925625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f3fcdbe0-9709-4ac3-94dc-5c963b80788c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:05.933616 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 697/1399] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:52:05.992042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da6128-25df-45c6-a5ea-0b9ec0de718b None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:05.992335 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da6128-25df-45c6-a5ea-0b9ec0de718b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:05.992613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da6128-25df-45c6-a5ea-0b9ec0de718b None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:05.994238 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 703/1400] 10.222.0.70 () {54 vars in 962 bytes} [Thu Jun 16 18:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:06.000298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850be11-9f48-48b7-a08c-e85bc2c7a99c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:06.000646 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850be11-9f48-48b7-a08c-e85bc2c7a99c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:06.001087 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850be11-9f48-48b7-a08c-e85bc2c7a99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:06.058834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0850be11-9f48-48b7-a08c-e85bc2c7a99c None None] MFA Rules not processed for user `8563a25d16134d639a2a8849976fed9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:06.060000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0850be11-9f48-48b7-a08c-e85bc2c7a99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:06.127926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 698/1401] 10.222.0.70 () {58 vars in 1059 bytes} [Thu Jun 16 18:52:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:06.136342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da6128-25df-45c6-a5ea-0b9ec0de718b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:06.148079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da6128-25df-45c6-a5ea-0b9ec0de718b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:06.153355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d661a-db62-4b6f-b946-66bca2cf5f8b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8563a25d16134d639a2a8849976fed9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:06.154021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d661a-db62-4b6f-b946-66bca2cf5f8b service glance] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:06.154314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d661a-db62-4b6f-b946-66bca2cf5f8b service glance] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:06.154637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d661a-db62-4b6f-b946-66bca2cf5f8b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:06.164598 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d661a-db62-4b6f-b946-66bca2cf5f8b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:06.166075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d661a-db62-4b6f-b946-66bca2cf5f8b service glance] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:06.176285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 704/1402] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:06.946161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb05e5-690f-458d-b8bd-53d78d5fbd51 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:06.946724 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb05e5-690f-458d-b8bd-53d78d5fbd51 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:06.946827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb05e5-690f-458d-b8bd-53d78d5fbd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:07.002983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-38cb05e5-690f-458d-b8bd-53d78d5fbd51 None None] MFA Rules not processed for user `02c826fc0baa480393df4c884c9e35dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:07.004311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-38cb05e5-690f-458d-b8bd-53d78d5fbd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:07.076709 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 699/1403] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Jun 16 18:52:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:52:07.120670 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43817f-8031-441a-901b-0cf83f8e2650 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:07.121383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43817f-8031-441a-901b-0cf83f8e2650 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:07.121383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43817f-8031-441a-901b-0cf83f8e2650 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:07.122670 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 705/1404] 10.222.0.70 () {54 vars in 942 bytes} [Thu Jun 16 18:52:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:07.129609 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88dcce-245f-48d1-b339-58dcd698163c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:07.129907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88dcce-245f-48d1-b339-58dcd698163c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:07.130184 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88dcce-245f-48d1-b339-58dcd698163c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:07.187282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ee88dcce-245f-48d1-b339-58dcd698163c None None] MFA Rules not processed for user `7b22fe0ab0354385924923d0d443acae`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:07.188961 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ee88dcce-245f-48d1-b339-58dcd698163c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:07.199050 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:52:07.268112 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 700/1405] 10.222.0.70 () {58 vars in 1039 bytes} [Thu Jun 16 18:52:07 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:07.278407 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43817f-8031-441a-901b-0cf83f8e2650 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:07.296148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43817f-8031-441a-901b-0cf83f8e2650 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:07.301103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7b5de-8bcf-4297-a90c-b236d1cc647b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b22fe0ab0354385924923d0d443acae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:07.301751 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7b5de-8bcf-4297-a90c-b236d1cc647b service swift] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:07.302036 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7b5de-8bcf-4297-a90c-b236d1cc647b service swift] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:07.302315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7b5de-8bcf-4297-a90c-b236d1cc647b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:07.312499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7b5de-8bcf-4297-a90c-b236d1cc647b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:07.314985 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7b5de-8bcf-4297-a90c-b236d1cc647b service swift] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:07.325323 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 706/1406] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Jun 16 18:52:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:52:08.180377 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50883eeb-829c-417d-a9e7-fb687747a5b4 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:08.180912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50883eeb-829c-417d-a9e7-fb687747a5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:08.180912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50883eeb-829c-417d-a9e7-fb687747a5b4 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:08.182873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 701/1407] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:52:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:08.188430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa4fb7-0173-47a0-9eee-3373a08aef46 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:08.188867 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa4fb7-0173-47a0-9eee-3373a08aef46 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:08.189008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa4fb7-0173-47a0-9eee-3373a08aef46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:08.212738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-84fa4fb7-0173-47a0-9eee-3373a08aef46 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:08.214461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-84fa4fb7-0173-47a0-9eee-3373a08aef46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:08.223045 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 707/1408] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:08.268255 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50883eeb-829c-417d-a9e7-fb687747a5b4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:08.284226 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50883eeb-829c-417d-a9e7-fb687747a5b4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:08.294161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ad275-9511-4bdf-93a7-2f0ecc6ce5bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:08.294925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ad275-9511-4bdf-93a7-2f0ecc6ce5bd service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:08.295336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ad275-9511-4bdf-93a7-2f0ecc6ce5bd service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:08.295615 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ad275-9511-4bdf-93a7-2f0ecc6ce5bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:08.304658 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ad275-9511-4bdf-93a7-2f0ecc6ce5bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:08.306341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ad275-9511-4bdf-93a7-2f0ecc6ce5bd service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:08.316350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 702/1409] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:18.202903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=79 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:52:18.311942 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=33 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:52:30.908666 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911ddda-08da-4aed-af65-822d05976037 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:30.909310 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911ddda-08da-4aed-af65-822d05976037 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:30.909310 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911ddda-08da-4aed-af65-822d05976037 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:30.910713 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 708/1410] 10.222.0.70 () {54 vars in 918 bytes} [Thu Jun 16 18:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:52:30.918565 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae6ec2-8e24-40e7-891c-0eb737ca1014 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:30.919321 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae6ec2-8e24-40e7-891c-0eb737ca1014 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:30.919321 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae6ec2-8e24-40e7-891c-0eb737ca1014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:30.937810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-79ae6ec2-8e24-40e7-891c-0eb737ca1014 None None] MFA Rules not processed for user `880d0299c38c4eb897c669a512e599be`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:30.939065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-79ae6ec2-8e24-40e7-891c-0eb737ca1014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:30.948696 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 703/1411] 10.222.0.70 () {58 vars in 1015 bytes} [Thu Jun 16 18:52:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:30.963190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d748701-bc9e-4195-a6bb-11b15b416017 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:30.964053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d748701-bc9e-4195-a6bb-11b15b416017 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:30.964535 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d748701-bc9e-4195-a6bb-11b15b416017 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:30.966729 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 709/1412] 10.222.0.70 () {54 vars in 918 bytes} [Thu Jun 16 18:52:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:30.972126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea777a-72fb-4ce9-9f0e-3a85aed8a32b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:30.972448 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea777a-72fb-4ce9-9f0e-3a85aed8a32b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:30.972775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea777a-72fb-4ce9-9f0e-3a85aed8a32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:30.996845 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-86ea777a-72fb-4ce9-9f0e-3a85aed8a32b None None] MFA Rules not processed for user `8342da838fe4462b9de2817ad3ac4ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:30.998886 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-86ea777a-72fb-4ce9-9f0e-3a85aed8a32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:31.013237 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 704/1413] 10.222.0.70 () {58 vars in 1015 bytes} [Thu Jun 16 18:52:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:33.737811 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ad1ad-2304-44cd-a59d-20f1a0806c37 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:33.738606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ad1ad-2304-44cd-a59d-20f1a0806c37 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:33.738606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ad1ad-2304-44cd-a59d-20f1a0806c37 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:33.740033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:33.747742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dae92-1925-4ddd-a0a1-b12aab264fee None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:33.748317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dae92-1925-4ddd-a0a1-b12aab264fee None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:33.748439 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dae92-1925-4ddd-a0a1-b12aab264fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:33.776445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-285dae92-1925-4ddd-a0a1-b12aab264fee None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:33.777675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-285dae92-1925-4ddd-a0a1-b12aab264fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:33.785844 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:33.833831 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ad1ad-2304-44cd-a59d-20f1a0806c37 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:33.848709 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ad1ad-2304-44cd-a59d-20f1a0806c37 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:33.854307 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e3283-2ee3-4c87-aa56-3d69a06637b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:33.855172 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e3283-2ee3-4c87-aa56-3d69a06637b9 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:33.855455 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e3283-2ee3-4c87-aa56-3d69a06637b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:33.855728 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e3283-2ee3-4c87-aa56-3d69a06637b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:33.864431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e3283-2ee3-4c87-aa56-3d69a06637b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:33.866015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e3283-2ee3-4c87-aa56-3d69a06637b9 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:33.876479 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 711/1416] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:34.888305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23be494-2e7d-4d90-91ad-4fb2964315c3 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:34.888868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23be494-2e7d-4d90-91ad-4fb2964315c3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:34.888868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23be494-2e7d-4d90-91ad-4fb2964315c3 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:34.890233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:34.896049 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b01ae-1039-4120-8b5d-c5a87810e2a3 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:34.896553 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b01ae-1039-4120-8b5d-c5a87810e2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:34.896785 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b01ae-1039-4120-8b5d-c5a87810e2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:34.916312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-733b01ae-1039-4120-8b5d-c5a87810e2a3 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:34.917800 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-733b01ae-1039-4120-8b5d-c5a87810e2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:34.925976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:52:35.087333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23be494-2e7d-4d90-91ad-4fb2964315c3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:35.102618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23be494-2e7d-4d90-91ad-4fb2964315c3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:35.108766 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0c14b-6aaf-4637-8600-a43c1409e6b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:35.109439 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0c14b-6aaf-4637-8600-a43c1409e6b3 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:35.109726 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0c14b-6aaf-4637-8600-a43c1409e6b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:35.110004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0c14b-6aaf-4637-8600-a43c1409e6b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:35.119901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0c14b-6aaf-4637-8600-a43c1409e6b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:35.121671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0c14b-6aaf-4637-8600-a43c1409e6b3 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:35.132443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 707/1419] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:36.097381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb8459-8457-4b35-97b8-8528cfbd61f9 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:36.097959 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb8459-8457-4b35-97b8-8528cfbd61f9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:36.097959 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb8459-8457-4b35-97b8-8528cfbd61f9 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:36.099412 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:36.104691 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fce42-c359-4159-b7b6-8569e93ee409 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:36.105159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fce42-c359-4159-b7b6-8569e93ee409 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:36.105546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fce42-c359-4159-b7b6-8569e93ee409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:36.124718 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-980fce42-c359-4159-b7b6-8569e93ee409 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:36.126042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-980fce42-c359-4159-b7b6-8569e93ee409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:36.134029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:36.291950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb8459-8457-4b35-97b8-8528cfbd61f9 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:36.306675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb8459-8457-4b35-97b8-8528cfbd61f9 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:36.313432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e3ec6-f147-4eb8-afa5-dd5a61e2dffb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:36.314085 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e3ec6-f147-4eb8-afa5-dd5a61e2dffb service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:36.314369 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e3ec6-f147-4eb8-afa5-dd5a61e2dffb service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:36.314678 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e3ec6-f147-4eb8-afa5-dd5a61e2dffb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:36.323868 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e3ec6-f147-4eb8-afa5-dd5a61e2dffb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:36.325474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e3ec6-f147-4eb8-afa5-dd5a61e2dffb service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:36.335519 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 714/1422] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:37.352322 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a41f5a-e933-40f1-81b6-62cee14be221 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:37.352886 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a41f5a-e933-40f1-81b6-62cee14be221 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:37.352886 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a41f5a-e933-40f1-81b6-62cee14be221 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:37.354275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:52:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:37.360980 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a25fd1-ad88-4aa8-b5d9-e5a6dbe00512 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:37.361944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a25fd1-ad88-4aa8-b5d9-e5a6dbe00512 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:37.361944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a25fd1-ad88-4aa8-b5d9-e5a6dbe00512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:37.387182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c2a25fd1-ad88-4aa8-b5d9-e5a6dbe00512 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:37.389261 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c2a25fd1-ad88-4aa8-b5d9-e5a6dbe00512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:37.400545 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:37.577817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a41f5a-e933-40f1-81b6-62cee14be221 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:37.604453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a41f5a-e933-40f1-81b6-62cee14be221 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:37.610675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e834472-6a7b-4190-b7d0-04ba17a464dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:37.611738 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e834472-6a7b-4190-b7d0-04ba17a464dd service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:37.612109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e834472-6a7b-4190-b7d0-04ba17a464dd service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:37.612384 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e834472-6a7b-4190-b7d0-04ba17a464dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:37.629778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e834472-6a7b-4190-b7d0-04ba17a464dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:37.631838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e834472-6a7b-4190-b7d0-04ba17a464dd service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:37.645802 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 710/1425] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:38.666767 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05178869-60fd-4461-9041-eccc278689ba None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:38.667673 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05178869-60fd-4461-9041-eccc278689ba None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:38.667673 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05178869-60fd-4461-9041-eccc278689ba None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:38.669267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:52:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:38.675500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd928c-a270-4065-9dca-f4464f860b56 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:38.676175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd928c-a270-4065-9dca-f4464f860b56 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:38.676175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd928c-a270-4065-9dca-f4464f860b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:38.701247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7edd928c-a270-4065-9dca-f4464f860b56 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:38.702465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7edd928c-a270-4065-9dca-f4464f860b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:38.710314 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:38.880224 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05178869-60fd-4461-9041-eccc278689ba None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:38.897221 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05178869-60fd-4461-9041-eccc278689ba None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:38.903799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0575569-29a8-402d-985d-c6df39f21494 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:38.904466 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0575569-29a8-402d-985d-c6df39f21494 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:38.904743 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0575569-29a8-402d-985d-c6df39f21494 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:38.905020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0575569-29a8-402d-985d-c6df39f21494 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:38.914364 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0575569-29a8-402d-985d-c6df39f21494 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:38.916105 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0575569-29a8-402d-985d-c6df39f21494 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:38.927347 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 717/1428] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:39.859374 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6080b-d475-4f37-8585-5c0f46683f6e None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:39.859917 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6080b-d475-4f37-8585-5c0f46683f6e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:39.859917 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6080b-d475-4f37-8585-5c0f46683f6e None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:39.861466 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:39.866615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8f0d8-637c-437a-ae73-13e5ef6bad5d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:39.867109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8f0d8-637c-437a-ae73-13e5ef6bad5d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:39.867226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8f0d8-637c-437a-ae73-13e5ef6bad5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:39.890959 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-aaa8f0d8-637c-437a-ae73-13e5ef6bad5d None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:39.892181 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-aaa8f0d8-637c-437a-ae73-13e5ef6bad5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:39.900828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:40.067336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6080b-d475-4f37-8585-5c0f46683f6e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:40.092495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6080b-d475-4f37-8585-5c0f46683f6e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:40.098426 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f323b00-851a-40e2-8c98-5cce9ea5fe04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:40.099108 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f323b00-851a-40e2-8c98-5cce9ea5fe04 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:40.099389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f323b00-851a-40e2-8c98-5cce9ea5fe04 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:40.099660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f323b00-851a-40e2-8c98-5cce9ea5fe04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:40.109344 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f323b00-851a-40e2-8c98-5cce9ea5fe04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:40.111035 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f323b00-851a-40e2-8c98-5cce9ea5fe04 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:40.121606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 713/1431] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:41.021419 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64f7bc-e0f8-4dc6-96a8-b2418416fd12 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:41.022005 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64f7bc-e0f8-4dc6-96a8-b2418416fd12 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:41.022102 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64f7bc-e0f8-4dc6-96a8-b2418416fd12 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:41.023609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:52:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:41.029823 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52208f-0eb7-4ffd-bdc3-c67a2cfd56f4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:41.030349 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52208f-0eb7-4ffd-bdc3-c67a2cfd56f4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:41.030447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52208f-0eb7-4ffd-bdc3-c67a2cfd56f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:41.052258 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-fe52208f-0eb7-4ffd-bdc3-c67a2cfd56f4 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:41.053451 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-fe52208f-0eb7-4ffd-bdc3-c67a2cfd56f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:41.065446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:41.229846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64f7bc-e0f8-4dc6-96a8-b2418416fd12 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:41.246142 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64f7bc-e0f8-4dc6-96a8-b2418416fd12 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:41.252483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9336e40-d1d0-4b96-9eb5-fe5126a7caa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:41.253150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9336e40-d1d0-4b96-9eb5-fe5126a7caa7 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:41.253436 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9336e40-d1d0-4b96-9eb5-fe5126a7caa7 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:41.253717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9336e40-d1d0-4b96-9eb5-fe5126a7caa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:41.263754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9336e40-d1d0-4b96-9eb5-fe5126a7caa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:41.265801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9336e40-d1d0-4b96-9eb5-fe5126a7caa7 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:41.278738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 720/1434] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:42.243935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa149b9-2dee-42d4-92f0-6da7f1956cb8 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:42.244509 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa149b9-2dee-42d4-92f0-6da7f1956cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:42.244509 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa149b9-2dee-42d4-92f0-6da7f1956cb8 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:42.245915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:42.252744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b48c3f-141f-4186-9108-1e16a5694264 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:42.253342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b48c3f-141f-4186-9108-1e16a5694264 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:42.253698 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b48c3f-141f-4186-9108-1e16a5694264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:42.279121 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-14b48c3f-141f-4186-9108-1e16a5694264 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:42.280729 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-14b48c3f-141f-4186-9108-1e16a5694264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:42.288373 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:42.447015 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa149b9-2dee-42d4-92f0-6da7f1956cb8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:42.463733 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa149b9-2dee-42d4-92f0-6da7f1956cb8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:42.469951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c7953-4958-4cf3-b83f-c24382f7501c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:42.470794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c7953-4958-4cf3-b83f-c24382f7501c service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:42.471175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c7953-4958-4cf3-b83f-c24382f7501c service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:42.471545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c7953-4958-4cf3-b83f-c24382f7501c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:42.481952 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c7953-4958-4cf3-b83f-c24382f7501c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:42.483641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c7953-4958-4cf3-b83f-c24382f7501c service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:42.494206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 716/1437] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:43.405521 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a09650-8082-48f9-99b6-d4945138a804 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:43.406056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a09650-8082-48f9-99b6-d4945138a804 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:43.406056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a09650-8082-48f9-99b6-d4945138a804 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:43.407379 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:43.412539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983caf78-a6bc-4f89-b79e-12abe283bb55 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:43.412941 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983caf78-a6bc-4f89-b79e-12abe283bb55 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:43.413103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983caf78-a6bc-4f89-b79e-12abe283bb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:43.437726 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-983caf78-a6bc-4f89-b79e-12abe283bb55 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:43.438972 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-983caf78-a6bc-4f89-b79e-12abe283bb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:43.446802 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:43.610364 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a09650-8082-48f9-99b6-d4945138a804 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:43.625876 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a09650-8082-48f9-99b6-d4945138a804 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:43.631693 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccf90d-3ddd-4109-bf9b-1f42927c1d97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:43.632527 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccf90d-3ddd-4109-bf9b-1f42927c1d97 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:43.632835 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccf90d-3ddd-4109-bf9b-1f42927c1d97 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:43.633115 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccf90d-3ddd-4109-bf9b-1f42927c1d97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:43.643432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccf90d-3ddd-4109-bf9b-1f42927c1d97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:43.645320 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccf90d-3ddd-4109-bf9b-1f42927c1d97 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:43.659261 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 723/1440] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:45.908346 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba45321-4eab-40a2-9fad-4c84dada9d5b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:45.908855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba45321-4eab-40a2-9fad-4c84dada9d5b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:45.908975 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba45321-4eab-40a2-9fad-4c84dada9d5b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:45.910321 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:52:45.915776 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ec762-c885-45e1-91f4-c56176b9da0b None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:45.916208 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ec762-c885-45e1-91f4-c56176b9da0b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:45.916417 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ec762-c885-45e1-91f4-c56176b9da0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:45.953456 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c67ec762-c885-45e1-91f4-c56176b9da0b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:45.954304 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c67ec762-c885-45e1-91f4-c56176b9da0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:45.964561 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:52:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:46.006521 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba45321-4eab-40a2-9fad-4c84dada9d5b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:46.021311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba45321-4eab-40a2-9fad-4c84dada9d5b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:46.025137 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f384982-336d-4d28-8c7e-c51f767cae38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:46.027091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f384982-336d-4d28-8c7e-c51f767cae38 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:46.027449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f384982-336d-4d28-8c7e-c51f767cae38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:46.027731 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f384982-336d-4d28-8c7e-c51f767cae38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:46.043800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f384982-336d-4d28-8c7e-c51f767cae38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:46.045263 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f384982-336d-4d28-8c7e-c51f767cae38 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:46.055843 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 719/1443] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:48.578315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d08e6-2a8a-4459-abc6-033a0f2b3384 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:48.578954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d08e6-2a8a-4459-abc6-033a0f2b3384 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:48.578954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d08e6-2a8a-4459-abc6-033a0f2b3384 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:48.580466 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:52:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:48.586142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f59c19-77d2-46e3-af65-61f9d2572e5e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:48.586621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f59c19-77d2-46e3-af65-61f9d2572e5e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:48.586765 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f59c19-77d2-46e3-af65-61f9d2572e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:48.608029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-67f59c19-77d2-46e3-af65-61f9d2572e5e None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:48.609233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-67f59c19-77d2-46e3-af65-61f9d2572e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:48.618151 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:48.658676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d08e6-2a8a-4459-abc6-033a0f2b3384 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:48.673299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d08e6-2a8a-4459-abc6-033a0f2b3384 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:48.677867 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff40fd-f2de-48ed-b1cb-189d46888333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:48.678600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff40fd-f2de-48ed-b1cb-189d46888333 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:48.678963 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff40fd-f2de-48ed-b1cb-189d46888333 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:48.679247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff40fd-f2de-48ed-b1cb-189d46888333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:48.692019 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff40fd-f2de-48ed-b1cb-189d46888333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:48.693475 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff40fd-f2de-48ed-b1cb-189d46888333 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:48.708146 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 726/1446] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:51.310326 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db74af8-1d0d-484b-8c92-4288682c7884 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:51.310920 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db74af8-1d0d-484b-8c92-4288682c7884 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:51.310920 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db74af8-1d0d-484b-8c92-4288682c7884 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:51.312292 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:51.317794 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baedb2-7899-465e-8436-dff26e51a60f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:51.318286 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baedb2-7899-465e-8436-dff26e51a60f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:51.318394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baedb2-7899-465e-8436-dff26e51a60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:51.336766 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-76baedb2-7899-465e-8436-dff26e51a60f None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:51.337968 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-76baedb2-7899-465e-8436-dff26e51a60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:51.347036 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:51.386725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db74af8-1d0d-484b-8c92-4288682c7884 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:51.388475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9db74af8-1d0d-484b-8c92-4288682c7884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:51.402453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db74af8-1d0d-484b-8c92-4288682c7884 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:51.406107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b1806-463d-4647-a8a3-1c0b04130f54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:51.406784 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b1806-463d-4647-a8a3-1c0b04130f54 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:51.407101 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b1806-463d-4647-a8a3-1c0b04130f54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:51.407379 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b1806-463d-4647-a8a3-1c0b04130f54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:51.416678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b1806-463d-4647-a8a3-1c0b04130f54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:51.418107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b1806-463d-4647-a8a3-1c0b04130f54 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:51.428435 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 722/1449] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:52:53.759381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a15c2-9261-473b-9faa-c07bf6d0a082 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:53.760498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a15c2-9261-473b-9faa-c07bf6d0a082 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:53.760498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a15c2-9261-473b-9faa-c07bf6d0a082 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:53.762063 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:52:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:53.767599 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a08597-fc82-499e-95b2-7b720217345f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:53.768050 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a08597-fc82-499e-95b2-7b720217345f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:53.768144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a08597-fc82-499e-95b2-7b720217345f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:53.787148 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-65a08597-fc82-499e-95b2-7b720217345f None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:53.788358 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-65a08597-fc82-499e-95b2-7b720217345f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:53.796391 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:53.835654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a15c2-9261-473b-9faa-c07bf6d0a082 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:53.846964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a15c2-9261-473b-9faa-c07bf6d0a082 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:53.850406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8e726-043f-4ee1-b669-ded078742de6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:53.851080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8e726-043f-4ee1-b669-ded078742de6 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:53.851366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8e726-043f-4ee1-b669-ded078742de6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:53.851643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8e726-043f-4ee1-b669-ded078742de6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:53.861661 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8e726-043f-4ee1-b669-ded078742de6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:53.863142 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8e726-043f-4ee1-b669-ded078742de6 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:53.876336 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 729/1452] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:56.145325 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692664ad-2002-4da7-8b6a-b8b718403352 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:56.146122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692664ad-2002-4da7-8b6a-b8b718403352 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:56.146122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692664ad-2002-4da7-8b6a-b8b718403352 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:56.147462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:56.153895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f4fb9-cbb8-4935-9fff-d13d94ea3310 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:56.154370 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f4fb9-cbb8-4935-9fff-d13d94ea3310 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:56.154503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f4fb9-cbb8-4935-9fff-d13d94ea3310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:56.173969 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5d3f4fb9-cbb8-4935-9fff-d13d94ea3310 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:56.175204 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5d3f4fb9-cbb8-4935-9fff-d13d94ea3310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:56.183325 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:52:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:56.221709 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692664ad-2002-4da7-8b6a-b8b718403352 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:56.233553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692664ad-2002-4da7-8b6a-b8b718403352 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:56.237110 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8762d8d-75a7-4368-b98a-72612a916a13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:56.237756 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8762d8d-75a7-4368-b98a-72612a916a13 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:56.238034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8762d8d-75a7-4368-b98a-72612a916a13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:56.238388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8762d8d-75a7-4368-b98a-72612a916a13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:56.248400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8762d8d-75a7-4368-b98a-72612a916a13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:56.250137 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8762d8d-75a7-4368-b98a-72612a916a13 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:56.260662 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 725/1455] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:58.558286 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ed563-19d0-4d7a-8aa0-34313733db09 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:58.558931 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ed563-19d0-4d7a-8aa0-34313733db09 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:58.558931 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ed563-19d0-4d7a-8aa0-34313733db09 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:58.560218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:58.565647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef8757-cc39-4e9d-aaa2-0fba5abdb972 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:58.566061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef8757-cc39-4e9d-aaa2-0fba5abdb972 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:58.566209 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef8757-cc39-4e9d-aaa2-0fba5abdb972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:58.572276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=70 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:52:58.586331 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9fef8757-cc39-4e9d-aaa2-0fba5abdb972 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:58.587617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9fef8757-cc39-4e9d-aaa2-0fba5abdb972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:58.596729 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:58.634840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ed563-19d0-4d7a-8aa0-34313733db09 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:58.641352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=64 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:52:58.648304 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ed563-19d0-4d7a-8aa0-34313733db09 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:58.652548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf3086-c43f-4e6f-9e4d-0ecf725c11bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:58.653189 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf3086-c43f-4e6f-9e4d-0ecf725c11bd service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:58.653470 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf3086-c43f-4e6f-9e4d-0ecf725c11bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:58.653745 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf3086-c43f-4e6f-9e4d-0ecf725c11bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:58.663252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf3086-c43f-4e6f-9e4d-0ecf725c11bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:58.664677 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf3086-c43f-4e6f-9e4d-0ecf725c11bd service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:58.674404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 732/1458] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:53:00.956604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c4e30-3980-4372-8db0-d5e91bb153bf None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:00.957101 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c4e30-3980-4372-8db0-d5e91bb153bf None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:00.957212 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c4e30-3980-4372-8db0-d5e91bb153bf None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:00.958549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:53:00.964825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a65150-bb3b-4416-99b1-88594e653e27 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:00.965318 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a65150-bb3b-4416-99b1-88594e653e27 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:00.965438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a65150-bb3b-4416-99b1-88594e653e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:00.984796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-02a65150-bb3b-4416-99b1-88594e653e27 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:53:00.986055 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-02a65150-bb3b-4416-99b1-88594e653e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:53:00.994250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:53:01.033206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c4e30-3980-4372-8db0-d5e91bb153bf None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:53:01.044921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c4e30-3980-4372-8db0-d5e91bb153bf None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:53:01.048549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae15bb4-c709-4bae-b0ab-bd7fda999ae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:53:01.049194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae15bb4-c709-4bae-b0ab-bd7fda999ae1 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:01.049478 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae15bb4-c709-4bae-b0ab-bd7fda999ae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:01.049751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae15bb4-c709-4bae-b0ab-bd7fda999ae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:01.060012 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae15bb4-c709-4bae-b0ab-bd7fda999ae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:53:01.061462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae15bb4-c709-4bae-b0ab-bd7fda999ae1 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:53:01.071429 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 728/1461] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:53:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:53:03.372782 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f80cd-f045-41f8-8a38-377e08fe1b25 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:03.373320 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f80cd-f045-41f8-8a38-377e08fe1b25 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:03.373320 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f80cd-f045-41f8-8a38-377e08fe1b25 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:03.374680 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:53:03.380240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761717f-c96d-4453-a74c-d37b31136e53 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:03.380657 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761717f-c96d-4453-a74c-d37b31136e53 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:03.380799 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761717f-c96d-4453-a74c-d37b31136e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:03.400569 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6761717f-c96d-4453-a74c-d37b31136e53 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:53:03.403316 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6761717f-c96d-4453-a74c-d37b31136e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:53:03.412888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:53:03.457337 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f80cd-f045-41f8-8a38-377e08fe1b25 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:53:03.471822 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f80cd-f045-41f8-8a38-377e08fe1b25 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:53:03.475443 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a52a3-b0d0-4cac-9beb-e30718fa537a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:53:03.476094 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a52a3-b0d0-4cac-9beb-e30718fa537a service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:03.476376 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a52a3-b0d0-4cac-9beb-e30718fa537a service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:03.476655 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a52a3-b0d0-4cac-9beb-e30718fa537a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:03.487023 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a52a3-b0d0-4cac-9beb-e30718fa537a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:53:03.488459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a52a3-b0d0-4cac-9beb-e30718fa537a service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:53:03.498345 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 735/1464] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:53:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:53:05.769115 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229afbe6-1250-40f2-bf8d-616beadc5b0a None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:05.769798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229afbe6-1250-40f2-bf8d-616beadc5b0a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:05.769798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229afbe6-1250-40f2-bf8d-616beadc5b0a None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:05.771341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:53:05.777681 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f89c8-a692-412d-ae1a-d9b7789c687b None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:05.778476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f89c8-a692-412d-ae1a-d9b7789c687b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:05.778476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f89c8-a692-412d-ae1a-d9b7789c687b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:05.798597 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2a9f89c8-a692-412d-ae1a-d9b7789c687b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:53:05.799826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2a9f89c8-a692-412d-ae1a-d9b7789c687b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:53:05.808946 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:53:05.846202 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229afbe6-1250-40f2-bf8d-616beadc5b0a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:53:05.856884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229afbe6-1250-40f2-bf8d-616beadc5b0a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:53:05.860476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9a5a5-a4f2-4080-8055-236d76ea8041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:53:05.861107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9a5a5-a4f2-4080-8055-236d76ea8041 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:05.861392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9a5a5-a4f2-4080-8055-236d76ea8041 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:05.861669 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9a5a5-a4f2-4080-8055-236d76ea8041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:05.871145 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9a5a5-a4f2-4080-8055-236d76ea8041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:53:05.872562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9a5a5-a4f2-4080-8055-236d76ea8041 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:53:05.882775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 731/1467] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:53:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:53:08.090855 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e2874-be4b-4bc2-9331-0cd331e9b816 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:08.091427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e2874-be4b-4bc2-9331-0cd331e9b816 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:08.091427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e2874-be4b-4bc2-9331-0cd331e9b816 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:08.092745 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:53:08.098497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be358f1e-f131-4410-a89c-1ccab2f12132 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:08.099002 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be358f1e-f131-4410-a89c-1ccab2f12132 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:08.099117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be358f1e-f131-4410-a89c-1ccab2f12132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:08.117661 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-be358f1e-f131-4410-a89c-1ccab2f12132 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:53:08.118906 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-be358f1e-f131-4410-a89c-1ccab2f12132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:53:08.128176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:53:08.169261 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e2874-be4b-4bc2-9331-0cd331e9b816 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:53:08.180595 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e2874-be4b-4bc2-9331-0cd331e9b816 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:53:08.185120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677ebc7-c243-4086-880c-7ec70699eb77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:53:08.185816 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677ebc7-c243-4086-880c-7ec70699eb77 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:08.186137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677ebc7-c243-4086-880c-7ec70699eb77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:08.186412 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677ebc7-c243-4086-880c-7ec70699eb77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:08.196579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677ebc7-c243-4086-880c-7ec70699eb77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:53:08.199411 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677ebc7-c243-4086-880c-7ec70699eb77 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:53:08.209493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 738/1470] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:53:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:53:18.112374 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=20 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:53:18.205078 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=23 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:53:29.338042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571655d0-2bbe-45a6-aaaa-8f1bf92d0e54 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:29.338647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571655d0-2bbe-45a6-aaaa-8f1bf92d0e54 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:29.338926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571655d0-2bbe-45a6-aaaa-8f1bf92d0e54 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:29.341393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 733/1471] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:53:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:53:29.347747 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83407b-879c-47d9-842c-d9c4c4519561 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:29.348482 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83407b-879c-47d9-842c-d9c4c4519561 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:29.348710 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83407b-879c-47d9-842c-d9c4c4519561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:29.372586 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5e83407b-879c-47d9-842c-d9c4c4519561 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:53:29.373795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5e83407b-879c-47d9-842c-d9c4c4519561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:53:29.381684 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 739/1472] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:53:29.431479 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d62038-75d5-45e7-ab0a-88917813656b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:29.431816 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d62038-75d5-45e7-ab0a-88917813656b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:29.433376 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d62038-75d5-45e7-ab0a-88917813656b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:29.434813 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 734/1473] 10.222.0.70 () {54 vars in 962 bytes} [Thu Jun 16 18:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:53:29.439816 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706aaa2d-987e-47b4-b6a5-f5e2c51dbf5f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:29.440109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706aaa2d-987e-47b4-b6a5-f5e2c51dbf5f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:29.440482 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706aaa2d-987e-47b4-b6a5-f5e2c51dbf5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:29.460190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-706aaa2d-987e-47b4-b6a5-f5e2c51dbf5f None None] MFA Rules not processed for user `8563a25d16134d639a2a8849976fed9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:53:29.461927 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-706aaa2d-987e-47b4-b6a5-f5e2c51dbf5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:53:29.469647 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 740/1474] 10.222.0.70 () {58 vars in 1059 bytes} [Thu Jun 16 18:53:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:53:29.477273 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d62038-75d5-45e7-ab0a-88917813656b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:53:29.496005 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d62038-75d5-45e7-ab0a-88917813656b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:53:29.499692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9262eb-2aa3-43c2-a1bb-7aad79dd6baa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8563a25d16134d639a2a8849976fed9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:53:29.500336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9262eb-2aa3-43c2-a1bb-7aad79dd6baa service glance] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:29.500618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9262eb-2aa3-43c2-a1bb-7aad79dd6baa service glance] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:29.500893 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9262eb-2aa3-43c2-a1bb-7aad79dd6baa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:29.512374 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9262eb-2aa3-43c2-a1bb-7aad79dd6baa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:53:29.513944 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9262eb-2aa3-43c2-a1bb-7aad79dd6baa service glance] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:53:29.524705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 735/1475] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:53:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:53:31.825212 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6afb71-caaa-43de-bbe1-907a102ac081 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:31.825814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6afb71-caaa-43de-bbe1-907a102ac081 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:31.825814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6afb71-caaa-43de-bbe1-907a102ac081 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:31.827165 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 741/1476] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:53:31.832592 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827327f-b5e8-44ff-b954-76edb14d33a0 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:31.833044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827327f-b5e8-44ff-b954-76edb14d33a0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:31.833162 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827327f-b5e8-44ff-b954-76edb14d33a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:31.851975 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b827327f-b5e8-44ff-b954-76edb14d33a0 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:53:31.853219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b827327f-b5e8-44ff-b954-76edb14d33a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:53:31.862397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 736/1477] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:53:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:53:31.886272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6afb71-caaa-43de-bbe1-907a102ac081 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:53:31.899934 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6afb71-caaa-43de-bbe1-907a102ac081 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:53:31.904513 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb5d86-78cb-41ae-b817-239b0b44b567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:53:31.905171 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb5d86-78cb-41ae-b817-239b0b44b567 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:31.905455 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb5d86-78cb-41ae-b817-239b0b44b567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:31.905734 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb5d86-78cb-41ae-b817-239b0b44b567 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:31.906473 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb5d86-78cb-41ae-b817-239b0b44b567 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:53:31.908543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: WARNING py.warnings [None req-ebeb5d86-78cb-41ae-b817-239b0b44b567 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:53:31.908543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: warnings.warn(msg) Jun 16 18:53:31.908543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: Jun 16 18:53:31.909425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb5d86-78cb-41ae-b817-239b0b44b567 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:53:31.931924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 742/1478] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Jun 16 18:53:31 2022] GET /identity/v3/projects => generated 2553 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:53:34.242690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed77fcd-3c44-4f39-bdad-b1d47ad58993 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:34.243281 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed77fcd-3c44-4f39-bdad-b1d47ad58993 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:34.243281 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed77fcd-3c44-4f39-bdad-b1d47ad58993 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:34.245183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 737/1479] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:53:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:53:34.251381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50919bf-eb48-4419-9b30-42531914c265 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:34.251974 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50919bf-eb48-4419-9b30-42531914c265 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:34.251974 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50919bf-eb48-4419-9b30-42531914c265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:34.271431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-a50919bf-eb48-4419-9b30-42531914c265 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:53:34.272942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a50919bf-eb48-4419-9b30-42531914c265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:53:34.281634 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 743/1480] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:53:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:53:34.328414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed77fcd-3c44-4f39-bdad-b1d47ad58993 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:53:34.343963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed77fcd-3c44-4f39-bdad-b1d47ad58993 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:53:34.350313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f6133-7ffb-4e77-a030-ccf3d01c4a4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:53:34.351015 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f6133-7ffb-4e77-a030-ccf3d01c4a4a service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:34.351300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f6133-7ffb-4e77-a030-ccf3d01c4a4a service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:34.351574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f6133-7ffb-4e77-a030-ccf3d01c4a4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:34.361672 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f6133-7ffb-4e77-a030-ccf3d01c4a4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:53:34.363309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f6133-7ffb-4e77-a030-ccf3d01c4a4a service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:53:34.374249 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 738/1481] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:53:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:53:36.906661 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d32fed-6f00-4460-9f4d-c2e45e5f8f35 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:36.907348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d32fed-6f00-4460-9f4d-c2e45e5f8f35 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:36.907348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d32fed-6f00-4460-9f4d-c2e45e5f8f35 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:36.908594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 744/1482] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:53:36.914301 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0f1bb-7dc1-4dc1-9be1-f51fd79d9303 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:36.914846 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0f1bb-7dc1-4dc1-9be1-f51fd79d9303 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:36.914954 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0f1bb-7dc1-4dc1-9be1-f51fd79d9303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:36.936089 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-06e0f1bb-7dc1-4dc1-9be1-f51fd79d9303 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:53:36.937315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-06e0f1bb-7dc1-4dc1-9be1-f51fd79d9303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:53:36.945422 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 739/1483] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:53:36.985616 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d32fed-6f00-4460-9f4d-c2e45e5f8f35 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:53:36.997739 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d32fed-6f00-4460-9f4d-c2e45e5f8f35 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:53:37.002988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff2d09-db1a-444d-bd10-13bfb0fb9ec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:53:37.003704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff2d09-db1a-444d-bd10-13bfb0fb9ec3 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:37.003918 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff2d09-db1a-444d-bd10-13bfb0fb9ec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:37.004194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff2d09-db1a-444d-bd10-13bfb0fb9ec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:37.016575 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff2d09-db1a-444d-bd10-13bfb0fb9ec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:53:37.018025 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff2d09-db1a-444d-bd10-13bfb0fb9ec3 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:53:37.029002 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 745/1484] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:53:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:53:39.300248 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b690a5-72dd-413c-9ecb-a7d2c7e7f0da None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:39.300850 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b690a5-72dd-413c-9ecb-a7d2c7e7f0da None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:39.300850 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b690a5-72dd-413c-9ecb-a7d2c7e7f0da None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:39.302421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 740/1485] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:53:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:53:39.309839 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4fa46-1dd8-4684-9bd0-9c33cb0d3092 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:39.310405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4fa46-1dd8-4684-9bd0-9c33cb0d3092 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:39.310504 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4fa46-1dd8-4684-9bd0-9c33cb0d3092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:39.332050 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-cec4fa46-1dd8-4684-9bd0-9c33cb0d3092 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:53:39.333256 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-cec4fa46-1dd8-4684-9bd0-9c33cb0d3092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:53:39.345031 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 746/1486] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:53:39.390456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b690a5-72dd-413c-9ecb-a7d2c7e7f0da None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:53:39.404741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b690a5-72dd-413c-9ecb-a7d2c7e7f0da None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:53:39.408479 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5d2af-774b-4877-a54f-d238316c9cdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:53:39.409123 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5d2af-774b-4877-a54f-d238316c9cdf service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:39.409403 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5d2af-774b-4877-a54f-d238316c9cdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:39.409675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5d2af-774b-4877-a54f-d238316c9cdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:39.420162 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5d2af-774b-4877-a54f-d238316c9cdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:53:39.421603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5d2af-774b-4877-a54f-d238316c9cdf service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:53:39.434078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 741/1487] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:53:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:53:41.858556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7095e-aca8-4b5e-afc1-d8f87006fa1b None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:41.859154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7095e-aca8-4b5e-afc1-d8f87006fa1b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:41.859154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7095e-aca8-4b5e-afc1-d8f87006fa1b None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:41.860521 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 747/1488] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:53:41.865799 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebefc6-36d8-4d56-ad12-3a1078033845 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:41.866236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebefc6-36d8-4d56-ad12-3a1078033845 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:41.866386 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebefc6-36d8-4d56-ad12-3a1078033845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:41.887309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-72ebefc6-36d8-4d56-ad12-3a1078033845 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:53:41.888546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-72ebefc6-36d8-4d56-ad12-3a1078033845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:53:41.896371 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 742/1489] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:53:41.937480 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7095e-aca8-4b5e-afc1-d8f87006fa1b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:53:41.949241 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7095e-aca8-4b5e-afc1-d8f87006fa1b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:53:41.952847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ee285-caf0-4f6b-845f-70ca241f9f09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:53:41.953486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ee285-caf0-4f6b-845f-70ca241f9f09 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:41.953768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ee285-caf0-4f6b-845f-70ca241f9f09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:41.954044 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ee285-caf0-4f6b-845f-70ca241f9f09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:41.963862 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ee285-caf0-4f6b-845f-70ca241f9f09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:53:41.965314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ee285-caf0-4f6b-845f-70ca241f9f09 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:53:41.975125 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 748/1490] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:53:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:53:44.393608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c60034-10ee-479b-9cce-d0cad42f3e04 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:44.394195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c60034-10ee-479b-9cce-d0cad42f3e04 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:44.394304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c60034-10ee-479b-9cce-d0cad42f3e04 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:44.395679 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 743/1491] 10.222.0.70 () {54 vars in 926 bytes} [Thu Jun 16 18:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:53:44.404151 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07525a16-0879-4417-b6ef-ccc574afebd7 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:44.404732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07525a16-0879-4417-b6ef-ccc574afebd7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:44.405132 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07525a16-0879-4417-b6ef-ccc574afebd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:44.433081 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-07525a16-0879-4417-b6ef-ccc574afebd7 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:53:44.434327 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-07525a16-0879-4417-b6ef-ccc574afebd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:53:44.442440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 749/1492] 10.222.0.70 () {58 vars in 1023 bytes} [Thu Jun 16 18:53:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:53:44.499768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c60034-10ee-479b-9cce-d0cad42f3e04 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:53:44.515902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c60034-10ee-479b-9cce-d0cad42f3e04 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:53:44.521621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c18d4-8352-48c0-9015-ca23c290d385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:53:44.522936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c18d4-8352-48c0-9015-ca23c290d385 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:44.523230 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c18d4-8352-48c0-9015-ca23c290d385 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:44.523512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c18d4-8352-48c0-9015-ca23c290d385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:44.533305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c18d4-8352-48c0-9015-ca23c290d385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:53:44.534770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c18d4-8352-48c0-9015-ca23c290d385 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:53:44.545773 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 744/1493] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:53:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:53:54.426606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=33 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:53:54.540186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=33 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:13.896015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d5959-ea1f-4e80-8024-d78c60ac6b70 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:13.896642 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d5959-ea1f-4e80-8024-d78c60ac6b70 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:13.896642 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d5959-ea1f-4e80-8024-d78c60ac6b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:13.917756 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2c9d5959-ea1f-4e80-8024-d78c60ac6b70 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:13.918980 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2c9d5959-ea1f-4e80-8024-d78c60ac6b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:13.926966 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 750/1494] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:55:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:13.936414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357234d-02b1-4f46-8312-e2f76f5bd353 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:13.936988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357234d-02b1-4f46-8312-e2f76f5bd353 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:13.936988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357234d-02b1-4f46-8312-e2f76f5bd353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:13.955366 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3357234d-02b1-4f46-8312-e2f76f5bd353 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:13.956581 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3357234d-02b1-4f46-8312-e2f76f5bd353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:13.965565 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 745/1495] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:55:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:13.970189 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d5959-ea1f-4e80-8024-d78c60ac6b70 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:13.981472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d5959-ea1f-4e80-8024-d78c60ac6b70 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:13.986772 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b61041-85b2-4330-b8e5-802feb343772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:13.987449 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b61041-85b2-4330-b8e5-802feb343772 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:13.987736 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b61041-85b2-4330-b8e5-802feb343772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:13.988018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b61041-85b2-4330-b8e5-802feb343772 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:13.988757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b61041-85b2-4330-b8e5-802feb343772 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:55:13.989072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b61041-85b2-4330-b8e5-802feb343772 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:13.990133 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: WARNING py.warnings [None req-79b61041-85b2-4330-b8e5-802feb343772 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:55:13.990133 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: warnings.warn(msg) Jun 16 18:55:13.990133 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: Jun 16 18:55:13.991078 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b61041-85b2-4330-b8e5-802feb343772 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:14.003787 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 751/1496] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:55:14.012587 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357234d-02b1-4f46-8312-e2f76f5bd353 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:14.022553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357234d-02b1-4f46-8312-e2f76f5bd353 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:14.029517 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907395c5-4242-4d57-8f61-d1d86d35e4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:14.031650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907395c5-4242-4d57-8f61-d1d86d35e4e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:14.031650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907395c5-4242-4d57-8f61-d1d86d35e4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:14.031650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907395c5-4242-4d57-8f61-d1d86d35e4e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:14.031971 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907395c5-4242-4d57-8f61-d1d86d35e4e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:14.033532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-907395c5-4242-4d57-8f61-d1d86d35e4e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:55:14.033532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jun 16 18:55:14.033532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: Jun 16 18:55:14.034807 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907395c5-4242-4d57-8f61-d1d86d35e4e4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:14.075415 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 746/1497] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:55:14 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:55:14.081231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b61041-85b2-4330-b8e5-802feb343772 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:14.097881 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b61041-85b2-4330-b8e5-802feb343772 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:14.158070 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433791c-af1b-457f-89ce-b82b26e76104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:14.158774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433791c-af1b-457f-89ce-b82b26e76104 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:14.159060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433791c-af1b-457f-89ce-b82b26e76104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:14.159357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433791c-af1b-457f-89ce-b82b26e76104 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:14.160031 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433791c-af1b-457f-89ce-b82b26e76104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:14.161067 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: WARNING py.warnings [None req-b433791c-af1b-457f-89ce-b82b26e76104 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:55:14.161067 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: warnings.warn(msg) Jun 16 18:55:14.161067 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: Jun 16 18:55:14.162258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433791c-af1b-457f-89ce-b82b26e76104 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:14.182977 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 752/1498] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:55:14 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:55:14.187311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907395c5-4242-4d57-8f61-d1d86d35e4e4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:14.198495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907395c5-4242-4d57-8f61-d1d86d35e4e4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:14.203139 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e15dd-d141-47eb-b2c2-ce0720592401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:14.203802 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e15dd-d141-47eb-b2c2-ce0720592401 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:14.204085 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e15dd-d141-47eb-b2c2-ce0720592401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:14.204362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e15dd-d141-47eb-b2c2-ce0720592401 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:14.205089 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e15dd-d141-47eb-b2c2-ce0720592401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:14.206103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-7c7e15dd-d141-47eb-b2c2-ce0720592401 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:55:14.206103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jun 16 18:55:14.206103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: Jun 16 18:55:14.207086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e15dd-d141-47eb-b2c2-ce0720592401 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:14.222643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 747/1499] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:14.226650 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433791c-af1b-457f-89ce-b82b26e76104 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:14.237855 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433791c-af1b-457f-89ce-b82b26e76104 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:14.242482 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb67eb2-8c41-4124-962a-214185e00782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:14.243859 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb67eb2-8c41-4124-962a-214185e00782 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:14.244141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb67eb2-8c41-4124-962a-214185e00782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:14.244417 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb67eb2-8c41-4124-962a-214185e00782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:14.245138 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb67eb2-8c41-4124-962a-214185e00782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:14.246159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: WARNING py.warnings [None req-aeb67eb2-8c41-4124-962a-214185e00782 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:55:14.246159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: warnings.warn(msg) Jun 16 18:55:14.246159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: Jun 16 18:55:14.247079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb67eb2-8c41-4124-962a-214185e00782 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:14.261833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 753/1500] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:14.266000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e15dd-d141-47eb-b2c2-ce0720592401 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:14.277294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e15dd-d141-47eb-b2c2-ce0720592401 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:14.281886 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f2a5d-7fda-4b1b-a1a1-028c76380a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:14.282810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f2a5d-7fda-4b1b-a1a1-028c76380a8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:14.283098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f2a5d-7fda-4b1b-a1a1-028c76380a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:14.283377 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f2a5d-7fda-4b1b-a1a1-028c76380a8f admin admin] PATH_INFO: `/v3/projects/1020b48a230b4847a3b4951dcba5e561/users/bc5d1e179adf496dace503f225b5aee3/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:14.298545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f2a5d-7fda-4b1b-a1a1-028c76380a8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1020b48a230b4847a3b4951dcba5e561, user_id=bc5d1e179adf496dace503f225b5aee3, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:14.299605 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-5d0f2a5d-7fda-4b1b-a1a1-028c76380a8f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:55:14.299605 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jun 16 18:55:14.299605 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: Jun 16 18:55:14.301531 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f2a5d-7fda-4b1b-a1a1-028c76380a8f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:14.315066 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 748/1501] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:55:14 2022] PUT /identity/v3/projects/1020b48a230b4847a3b4951dcba5e561/users/bc5d1e179adf496dace503f225b5aee3/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:14.327361 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d1b32-9967-446d-8bf9-ac1c1aff9302 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:14.327779 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d1b32-9967-446d-8bf9-ac1c1aff9302 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:14.328150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d1b32-9967-446d-8bf9-ac1c1aff9302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:14.362678 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-457d1b32-9967-446d-8bf9-ac1c1aff9302 None None] MFA Rules not processed for user `bc5d1e179adf496dace503f225b5aee3`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:14.363938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-457d1b32-9967-446d-8bf9-ac1c1aff9302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:14.440555 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 754/1502] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:55:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:14.455296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9aec64-ca0e-42d7-bcfc-5dd22888733a None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:14.455588 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9aec64-ca0e-42d7-bcfc-5dd22888733a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:14.456026 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9aec64-ca0e-42d7-bcfc-5dd22888733a None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:14.457277 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 749/1503] 10.222.0.70 () {54 vars in 963 bytes} [Thu Jun 16 18:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:55:14.463066 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1e28f-a403-4ec0-b31e-d7e9a36fbd4c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:14.463353 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1e28f-a403-4ec0-b31e-d7e9a36fbd4c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:14.463637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1e28f-a403-4ec0-b31e-d7e9a36fbd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:14.482082 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-50e1e28f-a403-4ec0-b31e-d7e9a36fbd4c None None] MFA Rules not processed for user `a47de73e3f3b4979bff83140bf287275`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:14.483328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-50e1e28f-a403-4ec0-b31e-d7e9a36fbd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:14.553814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:14.564865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 755/1504] 10.222.0.70 () {58 vars in 1060 bytes} [Thu Jun 16 18:55:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:55:14.578815 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9aec64-ca0e-42d7-bcfc-5dd22888733a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:14.593685 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9aec64-ca0e-42d7-bcfc-5dd22888733a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:14.597244 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d87e1-9364-4aff-a0a7-c44fe42705c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:14.597891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d87e1-9364-4aff-a0a7-c44fe42705c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:14.598174 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d87e1-9364-4aff-a0a7-c44fe42705c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:14.598449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d87e1-9364-4aff-a0a7-c44fe42705c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:14.608255 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d87e1-9364-4aff-a0a7-c44fe42705c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:14.609710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d87e1-9364-4aff-a0a7-c44fe42705c2 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:14.619751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 750/1505] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:55:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:55:14.662827 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc980ae-47d6-477c-ae9d-95049efac2b4 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:14.663145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc980ae-47d6-477c-ae9d-95049efac2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:14.663436 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc980ae-47d6-477c-ae9d-95049efac2b4 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:14.665462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 756/1506] 10.222.0.70 () {52 vars in 829 bytes} [Thu Jun 16 18:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:55:14.669430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d87e1-9364-4aff-a0a7-c44fe42705c2 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:14.740241 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d87e1-9364-4aff-a0a7-c44fe42705c2 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:14.746885 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331b78f-78b5-489c-b976-96cad0eae1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:14.747773 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331b78f-78b5-489c-b976-96cad0eae1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:14.748058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331b78f-78b5-489c-b976-96cad0eae1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:14.748340 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331b78f-78b5-489c-b976-96cad0eae1d4 admin admin] PATH_INFO: `/v3/users/bc5d1e179adf496dace503f225b5aee3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:14.749590 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331b78f-78b5-489c-b976-96cad0eae1d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc5d1e179adf496dace503f225b5aee3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:14.750602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-a331b78f-78b5-489c-b976-96cad0eae1d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:55:14.750602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jun 16 18:55:14.750602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: Jun 16 18:55:14.751954 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331b78f-78b5-489c-b976-96cad0eae1d4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:14.841091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a331b78f-78b5-489c-b976-96cad0eae1d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc5d1e179adf496dace503f225b5aee3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:14.841764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a331b78f-78b5-489c-b976-96cad0eae1d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc5d1e179adf496dace503f225b5aee3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:14.842535 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a331b78f-78b5-489c-b976-96cad0eae1d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc5d1e179adf496dace503f225b5aee3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:14.857423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a331b78f-78b5-489c-b976-96cad0eae1d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc5d1e179adf496dace503f225b5aee3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:14.871471 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a331b78f-78b5-489c-b976-96cad0eae1d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc5d1e179adf496dace503f225b5aee3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:14.891196 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:14.901808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 751/1507] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:55:14 2022] DELETE /identity/v3/users/bc5d1e179adf496dace503f225b5aee3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:14.906275 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d5254-0fd2-4d3a-9611-37b3441fa1b9 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:14.906843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d5254-0fd2-4d3a-9611-37b3441fa1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:14.906952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d5254-0fd2-4d3a-9611-37b3441fa1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:14.927970 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c47d5254-0fd2-4d3a-9611-37b3441fa1b9 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:14.929285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c47d5254-0fd2-4d3a-9611-37b3441fa1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:14.994302 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 757/1508] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:15.004068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331b78f-78b5-489c-b976-96cad0eae1d4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:15.005586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a331b78f-78b5-489c-b976-96cad0eae1d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:15.064872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331b78f-78b5-489c-b976-96cad0eae1d4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:15.068496 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65797a-0a47-482f-9263-31072eb9cccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:15.069153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65797a-0a47-482f-9263-31072eb9cccd service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:15.069439 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65797a-0a47-482f-9263-31072eb9cccd service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:15.069718 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65797a-0a47-482f-9263-31072eb9cccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:15.079823 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65797a-0a47-482f-9263-31072eb9cccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:15.081284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65797a-0a47-482f-9263-31072eb9cccd service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:15.092095 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 752/1509] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:55:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:55:15.439788 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d5254-0fd2-4d3a-9611-37b3441fa1b9 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:15.441466 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c47d5254-0fd2-4d3a-9611-37b3441fa1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:15.455349 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d5254-0fd2-4d3a-9611-37b3441fa1b9 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:15.465765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e7750-780f-451e-abf6-58a7d73794a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:15.467155 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e7750-780f-451e-abf6-58a7d73794a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:15.467439 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e7750-780f-451e-abf6-58a7d73794a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:15.467711 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e7750-780f-451e-abf6-58a7d73794a9 admin admin] PATH_INFO: `/v3/projects/1020b48a230b4847a3b4951dcba5e561` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:15.468996 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e7750-780f-451e-abf6-58a7d73794a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1020b48a230b4847a3b4951dcba5e561)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:15.470029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: WARNING py.warnings [None req-e60e7750-780f-451e-abf6-58a7d73794a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:55:15.470029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: warnings.warn(msg) Jun 16 18:55:15.470029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: Jun 16 18:55:15.471277 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e7750-780f-451e-abf6-58a7d73794a9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:15.546362 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e60e7750-780f-451e-abf6-58a7d73794a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1020b48a230b4847a3b4951dcba5e561', 'request_id': 'req-e60e7750-780f-451e-abf6-58a7d73794a9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:15.558317 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e60e7750-780f-451e-abf6-58a7d73794a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1020b48a230b4847a3b4951dcba5e561', 'request_id': 'req-e60e7750-780f-451e-abf6-58a7d73794a9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:15.578247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e60e7750-780f-451e-abf6-58a7d73794a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1020b48a230b4847a3b4951dcba5e561', 'request_id': 'req-e60e7750-780f-451e-abf6-58a7d73794a9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:15.593866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e60e7750-780f-451e-abf6-58a7d73794a9 admin admin] The token cache is being invalidate because project 1020b48a230b4847a3b4951dcba5e561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:55:15.593866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e60e7750-780f-451e-abf6-58a7d73794a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:15.593866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e60e7750-780f-451e-abf6-58a7d73794a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:15.597224 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 758/1510] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 18:55:15 2022] DELETE /identity/v3/projects/1020b48a230b4847a3b4951dcba5e561 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:25.087122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:25.594006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:03.166979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3248c-667a-4d7e-b425-da1163dfdf72 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:03.167590 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3248c-667a-4d7e-b425-da1163dfdf72 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:03.167590 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3248c-667a-4d7e-b425-da1163dfdf72 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:03.168973 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:57:03.174512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ea1b6-5b85-429b-aa02-eba28c133357 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:03.174959 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ea1b6-5b85-429b-aa02-eba28c133357 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:03.175111 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ea1b6-5b85-429b-aa02-eba28c133357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:03.193472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-7e7ea1b6-5b85-429b-aa02-eba28c133357 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:03.194786 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7e7ea1b6-5b85-429b-aa02-eba28c133357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:03.255411 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:57:03.429899 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3248c-667a-4d7e-b425-da1163dfdf72 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:03.431715 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-aae3248c-667a-4d7e-b425-da1163dfdf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:03.510614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3248c-667a-4d7e-b425-da1163dfdf72 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:03.516023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d2efc-6b6c-4712-a2e6-946a80b97119 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:03.516683 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d2efc-6b6c-4712-a2e6-946a80b97119 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:03.516970 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d2efc-6b6c-4712-a2e6-946a80b97119 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:03.517249 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d2efc-6b6c-4712-a2e6-946a80b97119 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:03.527160 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d2efc-6b6c-4712-a2e6-946a80b97119 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:03.528779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d2efc-6b6c-4712-a2e6-946a80b97119 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:03.538879 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 754/1513] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:04.458394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e6952-1f35-4990-8480-70ce34b9c291 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:04.459553 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e6952-1f35-4990-8480-70ce34b9c291 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:04.459553 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e6952-1f35-4990-8480-70ce34b9c291 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:04.460938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:57:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:57:04.466102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2fe0e-cca7-4c3b-9a4e-b6e1db174f24 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:04.466607 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2fe0e-cca7-4c3b-9a4e-b6e1db174f24 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:04.466713 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2fe0e-cca7-4c3b-9a4e-b6e1db174f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:04.486871 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6ba2fe0e-cca7-4c3b-9a4e-b6e1db174f24 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:04.488158 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6ba2fe0e-cca7-4c3b-9a4e-b6e1db174f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:04.496748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:04.652220 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e6952-1f35-4990-8480-70ce34b9c291 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:04.668324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e6952-1f35-4990-8480-70ce34b9c291 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:04.673667 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507025f-e45a-4ca5-9535-32e998bb88aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:04.674302 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507025f-e45a-4ca5-9535-32e998bb88aa service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:04.674599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507025f-e45a-4ca5-9535-32e998bb88aa service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:04.674886 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507025f-e45a-4ca5-9535-32e998bb88aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:04.683953 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6507025f-e45a-4ca5-9535-32e998bb88aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:04.685559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6507025f-e45a-4ca5-9535-32e998bb88aa service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:04.695918 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 761/1516] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:06.473588 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307134b-7f4a-41d0-a326-64bd645f452c None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:06.474267 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307134b-7f4a-41d0-a326-64bd645f452c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:06.474267 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307134b-7f4a-41d0-a326-64bd645f452c None None] PATH_INFO: `/v3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:06.475538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 756/1517] 10.222.0.78 () {54 vars in 925 bytes} [Thu Jun 16 18:57:06 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:57:06.482178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb20726-ffb6-4aed-8f7a-912ca9b24d41 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:06.482723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb20726-ffb6-4aed-8f7a-912ca9b24d41 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:06.482828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb20726-ffb6-4aed-8f7a-912ca9b24d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:06.501753 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ecb20726-ffb6-4aed-8f7a-912ca9b24d41 None None] MFA Rules not processed for user `880d0299c38c4eb897c669a512e599be`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:06.503262 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ecb20726-ffb6-4aed-8f7a-912ca9b24d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:06.568950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 762/1518] 10.222.0.78 () {58 vars in 1015 bytes} [Thu Jun 16 18:57:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:10.644466 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301e756-ada9-493c-8b44-a6b5c6d8c4d2 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:10.644997 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301e756-ada9-493c-8b44-a6b5c6d8c4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:10.645103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301e756-ada9-493c-8b44-a6b5c6d8c4d2 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:10.647137 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 757/1519] 10.222.0.70 () {54 vars in 960 bytes} [Thu Jun 16 18:57:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:57:10.655797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87deb1-c08a-4bfb-aacb-b88913ba6e15 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:10.656341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87deb1-c08a-4bfb-aacb-b88913ba6e15 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:10.656341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87deb1-c08a-4bfb-aacb-b88913ba6e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:10.682428 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-0b87deb1-c08a-4bfb-aacb-b88913ba6e15 None None] MFA Rules not processed for user `880d0299c38c4eb897c669a512e599be`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:10.683684 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0b87deb1-c08a-4bfb-aacb-b88913ba6e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:10.692212 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 763/1520] 10.222.0.70 () {58 vars in 1057 bytes} [Thu Jun 16 18:57:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:10.701650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301e756-ada9-493c-8b44-a6b5c6d8c4d2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:10.716515 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301e756-ada9-493c-8b44-a6b5c6d8c4d2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:10.721675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedb059-4ce1-44f7-b05d-6fa8bc50c36e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '880d0299c38c4eb897c669a512e599be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:10.722328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedb059-4ce1-44f7-b05d-6fa8bc50c36e service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:10.722771 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedb059-4ce1-44f7-b05d-6fa8bc50c36e service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:10.723070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedb059-4ce1-44f7-b05d-6fa8bc50c36e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:10.734041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceedb059-4ce1-44f7-b05d-6fa8bc50c36e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:10.735892 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceedb059-4ce1-44f7-b05d-6fa8bc50c36e service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:10.748947 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 758/1521] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:16.052117 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bc5b7-c3ca-4ca2-bad9-37707ec6258a None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:16.053246 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bc5b7-c3ca-4ca2-bad9-37707ec6258a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:16.053246 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bc5b7-c3ca-4ca2-bad9-37707ec6258a None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:16.054591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:57:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:57:16.059717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77cac2-313c-4626-944c-af7eb079a923 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:16.060170 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77cac2-313c-4626-944c-af7eb079a923 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:16.060269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77cac2-313c-4626-944c-af7eb079a923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:16.078623 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4a77cac2-313c-4626-944c-af7eb079a923 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:16.079817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4a77cac2-313c-4626-944c-af7eb079a923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:16.087890 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:16.246623 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bc5b7-c3ca-4ca2-bad9-37707ec6258a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:16.261796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bc5b7-c3ca-4ca2-bad9-37707ec6258a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:16.267124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256213d-daf7-4e87-ac1b-3716615bdce3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:16.267766 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256213d-daf7-4e87-ac1b-3716615bdce3 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:16.268047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256213d-daf7-4e87-ac1b-3716615bdce3 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:16.268323 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256213d-daf7-4e87-ac1b-3716615bdce3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:16.277729 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256213d-daf7-4e87-ac1b-3716615bdce3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:16.279381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256213d-daf7-4e87-ac1b-3716615bdce3 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:16.290410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 765/1524] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:17.207789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f3936-1927-4e68-8112-bc725f92356f None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:17.208926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f3936-1927-4e68-8112-bc725f92356f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:17.208926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f3936-1927-4e68-8112-bc725f92356f None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:17.210665 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:57:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:57:17.216033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fa95b-c850-4dd8-8ae5-d9c1623487ff None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:17.216506 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fa95b-c850-4dd8-8ae5-d9c1623487ff None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:17.216624 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fa95b-c850-4dd8-8ae5-d9c1623487ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:17.236079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c80fa95b-c850-4dd8-8ae5-d9c1623487ff None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:17.237299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c80fa95b-c850-4dd8-8ae5-d9c1623487ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:17.246153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:17.403909 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f3936-1927-4e68-8112-bc725f92356f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:17.418825 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f3936-1927-4e68-8112-bc725f92356f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:17.424185 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ee3a7-6f2e-46f0-b799-3f3ce9063b3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:17.424834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ee3a7-6f2e-46f0-b799-3f3ce9063b3f service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:17.425120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ee3a7-6f2e-46f0-b799-3f3ce9063b3f service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:17.425399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ee3a7-6f2e-46f0-b799-3f3ce9063b3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:17.435426 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ee3a7-6f2e-46f0-b799-3f3ce9063b3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:17.437042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ee3a7-6f2e-46f0-b799-3f3ce9063b3f service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:17.448208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 761/1527] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:21.966651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3a572-2353-4b88-b6df-31dbd09fa8fc None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:21.967207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3a572-2353-4b88-b6df-31dbd09fa8fc None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:21.967207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3a572-2353-4b88-b6df-31dbd09fa8fc None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:21.968647 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:57:21.974791 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196b3f3-d648-4a89-ab21-40c66b182c7a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:21.974791 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196b3f3-d648-4a89-ab21-40c66b182c7a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:21.974791 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196b3f3-d648-4a89-ab21-40c66b182c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:21.994122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a196b3f3-d648-4a89-ab21-40c66b182c7a None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:21.995375 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a196b3f3-d648-4a89-ab21-40c66b182c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:22.003262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:22.040456 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3a572-2353-4b88-b6df-31dbd09fa8fc None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:22.042173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-fcb3a572-2353-4b88-b6df-31dbd09fa8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:22.101268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3a572-2353-4b88-b6df-31dbd09fa8fc None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:22.104943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5b7d9-f6b6-459b-8ad2-5c7382a44523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:22.105588 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5b7d9-f6b6-459b-8ad2-5c7382a44523 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:22.105868 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5b7d9-f6b6-459b-8ad2-5c7382a44523 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:22.106148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5b7d9-f6b6-459b-8ad2-5c7382a44523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:22.115690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5b7d9-f6b6-459b-8ad2-5c7382a44523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:22.117135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5b7d9-f6b6-459b-8ad2-5c7382a44523 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:22.122293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:22.127652 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 768/1530] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:24.298263 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec83532-9ac5-45ef-9dfc-e7c6c92fe172 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:24.298806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec83532-9ac5-45ef-9dfc-e7c6c92fe172 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:24.298806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec83532-9ac5-45ef-9dfc-e7c6c92fe172 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:24.300133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:57:24.306271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e258e-c2b1-4192-9e80-a9a213876222 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:24.306800 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e258e-c2b1-4192-9e80-a9a213876222 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:24.306911 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e258e-c2b1-4192-9e80-a9a213876222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:24.325496 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-0d1e258e-c2b1-4192-9e80-a9a213876222 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:24.326741 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0d1e258e-c2b1-4192-9e80-a9a213876222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:24.335135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:24.372102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec83532-9ac5-45ef-9dfc-e7c6c92fe172 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:24.373889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0ec83532-9ac5-45ef-9dfc-e7c6c92fe172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:24.387667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec83532-9ac5-45ef-9dfc-e7c6c92fe172 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:24.391183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51074d-b1a9-4c41-a97f-e5ddfcdd6111 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:24.391832 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51074d-b1a9-4c41-a97f-e5ddfcdd6111 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:24.392151 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51074d-b1a9-4c41-a97f-e5ddfcdd6111 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:24.392381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51074d-b1a9-4c41-a97f-e5ddfcdd6111 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:24.402509 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51074d-b1a9-4c41-a97f-e5ddfcdd6111 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:24.403971 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51074d-b1a9-4c41-a97f-e5ddfcdd6111 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:24.414834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 764/1533] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:28.465131 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894631f3-27bc-45d2-b494-a6d9827f8c02 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:28.465709 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894631f3-27bc-45d2-b494-a6d9827f8c02 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:28.465709 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894631f3-27bc-45d2-b494-a6d9827f8c02 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:28.467287 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:57:28.472962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a20dc7-2fa9-4177-b0e0-4380d232deb9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:28.473446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a20dc7-2fa9-4177-b0e0-4380d232deb9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:28.473625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a20dc7-2fa9-4177-b0e0-4380d232deb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:28.480256 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=62 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:28.493927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d8a20dc7-2fa9-4177-b0e0-4380d232deb9 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:28.495176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d8a20dc7-2fa9-4177-b0e0-4380d232deb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:28.503312 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:57:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:28.783522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894631f3-27bc-45d2-b494-a6d9827f8c02 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:28.785236 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-894631f3-27bc-45d2-b494-a6d9827f8c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:28.848256 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894631f3-27bc-45d2-b494-a6d9827f8c02 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:28.853022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f93409-e0db-4ab5-b1e6-10026368c314 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:28.853677 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f93409-e0db-4ab5-b1e6-10026368c314 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:28.853962 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f93409-e0db-4ab5-b1e6-10026368c314 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:28.854242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f93409-e0db-4ab5-b1e6-10026368c314 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:28.866746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f93409-e0db-4ab5-b1e6-10026368c314 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:28.868204 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f93409-e0db-4ab5-b1e6-10026368c314 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:28.878277 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 771/1536] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:31.158326 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25264e1f-5101-4026-b613-3198d3307640 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:31.158923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25264e1f-5101-4026-b613-3198d3307640 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:31.158923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25264e1f-5101-4026-b613-3198d3307640 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:31.160203 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:57:31.165899 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f743edf-ca15-4ffe-a965-f2d74cc9c71b None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:31.166391 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f743edf-ca15-4ffe-a965-f2d74cc9c71b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:31.166506 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f743edf-ca15-4ffe-a965-f2d74cc9c71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:31.184641 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1f743edf-ca15-4ffe-a965-f2d74cc9c71b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:31.185948 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1f743edf-ca15-4ffe-a965-f2d74cc9c71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:31.194625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:31.462800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25264e1f-5101-4026-b613-3198d3307640 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:31.464494 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-25264e1f-5101-4026-b613-3198d3307640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:31.481743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25264e1f-5101-4026-b613-3198d3307640 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:31.485727 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737587b-d8dd-416c-be4d-e2271938b2e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:31.486689 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737587b-d8dd-416c-be4d-e2271938b2e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:31.487220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737587b-d8dd-416c-be4d-e2271938b2e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:31.487711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737587b-d8dd-416c-be4d-e2271938b2e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:31.507133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1737587b-d8dd-416c-be4d-e2271938b2e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:31.508608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1737587b-d8dd-416c-be4d-e2271938b2e8 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:31.521626 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 767/1539] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:57:41.179426 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=32 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:41.516717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=8 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:05.467257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee877f-37c6-4eed-b382-3a829b197e94 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:05.468305 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee877f-37c6-4eed-b382-3a829b197e94 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:05.468759 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee877f-37c6-4eed-b382-3a829b197e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:05.499207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-68ee877f-37c6-4eed-b382-3a829b197e94 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:05.500758 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-68ee877f-37c6-4eed-b382-3a829b197e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:05.510637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 773/1540] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:05.521448 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48690807-f2e1-43f5-b1a6-2e4382f12369 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:05.522699 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48690807-f2e1-43f5-b1a6-2e4382f12369 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:05.522699 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48690807-f2e1-43f5-b1a6-2e4382f12369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:05.557024 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-48690807-f2e1-43f5-b1a6-2e4382f12369 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:05.558363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-48690807-f2e1-43f5-b1a6-2e4382f12369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:05.574028 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 768/1541] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:05.578353 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee877f-37c6-4eed-b382-3a829b197e94 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:05.593960 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee877f-37c6-4eed-b382-3a829b197e94 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:05.606381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a4342-b665-46b1-ad38-960e886cb1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:05.607479 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a4342-b665-46b1-ad38-960e886cb1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:05.607984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a4342-b665-46b1-ad38-960e886cb1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:05.608486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a4342-b665-46b1-ad38-960e886cb1d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:05.609438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a4342-b665-46b1-ad38-960e886cb1d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:59:05.610017 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a4342-b665-46b1-ad38-960e886cb1d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:05.612171 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a4342-b665-46b1-ad38-960e886cb1d8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:05.632904 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 774/1542] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:59:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:59:05.641479 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48690807-f2e1-43f5-b1a6-2e4382f12369 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:05.655080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48690807-f2e1-43f5-b1a6-2e4382f12369 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:05.663720 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabedbf4-760b-48b3-96bf-8bc70dc8dbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:05.664407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabedbf4-760b-48b3-96bf-8bc70dc8dbaa admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:05.664692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabedbf4-760b-48b3-96bf-8bc70dc8dbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:05.664971 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabedbf4-760b-48b3-96bf-8bc70dc8dbaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:05.665825 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabedbf4-760b-48b3-96bf-8bc70dc8dbaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:05.667884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabedbf4-760b-48b3-96bf-8bc70dc8dbaa admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:05.712058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 769/1543] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:05 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:05.715501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a4342-b665-46b1-ad38-960e886cb1d8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:05.730530 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a4342-b665-46b1-ad38-960e886cb1d8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:05.818004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2ee51-ce81-4c8d-975e-3046da3e0ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:05.819113 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2ee51-ce81-4c8d-975e-3046da3e0ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:05.819623 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2ee51-ce81-4c8d-975e-3046da3e0ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:05.820145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2ee51-ce81-4c8d-975e-3046da3e0ceb admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:05.821054 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2ee51-ce81-4c8d-975e-3046da3e0ceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:05.823279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2ee51-ce81-4c8d-975e-3046da3e0ceb admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:05.844896 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d60be0-3699-4920-9dba-d33967f6422b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:05.846255 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d60be0-3699-4920-9dba-d33967f6422b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:05.846778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d60be0-3699-4920-9dba-d33967f6422b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:05.857713 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 775/1544] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:05 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:05.862918 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2ee51-ce81-4c8d-975e-3046da3e0ceb admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:05.879107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2ee51-ce81-4c8d-975e-3046da3e0ceb admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:05.883603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-96d60be0-3699-4920-9dba-d33967f6422b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:05.884838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-96d60be0-3699-4920-9dba-d33967f6422b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:05.888944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6785f-25f2-4257-ae48-0c87c919bee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:05.889907 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6785f-25f2-4257-ae48-0c87c919bee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:05.890457 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6785f-25f2-4257-ae48-0c87c919bee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:05.890967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6785f-25f2-4257-ae48-0c87c919bee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:05.891924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f6785f-25f2-4257-ae48-0c87c919bee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:05.894400 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f6785f-25f2-4257-ae48-0c87c919bee7 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:05.898872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 770/1545] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:59:05.910137 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b66cd-208e-4954-b092-dc5520c83d00 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:05.910514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b66cd-208e-4954-b092-dc5520c83d00 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:05.910875 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b66cd-208e-4954-b092-dc5520c83d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:05.913290 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 776/1546] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:59:05.916950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6785f-25f2-4257-ae48-0c87c919bee7 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:05.933158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6785f-25f2-4257-ae48-0c87c919bee7 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:05.944961 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-637b66cd-208e-4954-b092-dc5520c83d00 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:05.946186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-637b66cd-208e-4954-b092-dc5520c83d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:05.950879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc865eca-bfd6-4446-ade9-c7a8093f2776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:05.951952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc865eca-bfd6-4446-ade9-c7a8093f2776 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:05.953187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc865eca-bfd6-4446-ade9-c7a8093f2776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:05.953695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc865eca-bfd6-4446-ade9-c7a8093f2776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:05.954726 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc865eca-bfd6-4446-ade9-c7a8093f2776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:05.957156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc865eca-bfd6-4446-ade9-c7a8093f2776 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:05.966227 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 771/1547] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:05.968438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262787c9-74fe-418f-aa31-97a26e9dade2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:05.974747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262787c9-74fe-418f-aa31-97a26e9dade2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:05.974747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262787c9-74fe-418f-aa31-97a26e9dade2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:05.986878 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 777/1548] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:05.988580 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e006c-4cc9-4315-a3c5-dc7c6d3a8fa8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:05.989092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e006c-4cc9-4315-a3c5-dc7c6d3a8fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:05.989585 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e006c-4cc9-4315-a3c5-dc7c6d3a8fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.008458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-262787c9-74fe-418f-aa31-97a26e9dade2 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:06.010219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-262787c9-74fe-418f-aa31-97a26e9dade2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:06.037297 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 772/1549] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:06.037297 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262787c9-74fe-418f-aa31-97a26e9dade2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.047741 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-230e006c-4cc9-4315-a3c5-dc7c6d3a8fa8 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:06.050394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-230e006c-4cc9-4315-a3c5-dc7c6d3a8fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:06.056727 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262787c9-74fe-418f-aa31-97a26e9dade2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:06.063615 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335ef66-04a6-411d-9649-df11e0b234ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:06.064919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335ef66-04a6-411d-9649-df11e0b234ee admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.065350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335ef66-04a6-411d-9649-df11e0b234ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.065962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335ef66-04a6-411d-9649-df11e0b234ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.066798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335ef66-04a6-411d-9649-df11e0b234ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:59:06.067114 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335ef66-04a6-411d-9649-df11e0b234ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:06.068284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-a335ef66-04a6-411d-9649-df11e0b234ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:59:06.068284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jun 16 18:59:06.068284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: Jun 16 18:59:06.069227 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335ef66-04a6-411d-9649-df11e0b234ee admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:06.073130 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 778/1550] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:06.074253 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e006c-4cc9-4315-a3c5-dc7c6d3a8fa8 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.084844 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e006c-4cc9-4315-a3c5-dc7c6d3a8fa8 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:06.096200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8aaec-0bc0-4e1d-b46f-e794ac0f8b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:06.096865 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 773/1551] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:59:06.097754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8aaec-0bc0-4e1d-b46f-e794ac0f8b3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.098824 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e525f5a-4ec0-4890-9105-1765744e8a9a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.099059 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8aaec-0bc0-4e1d-b46f-e794ac0f8b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.099198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e525f5a-4ec0-4890-9105-1765744e8a9a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.099407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e525f5a-4ec0-4890-9105-1765744e8a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.100008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8aaec-0bc0-4e1d-b46f-e794ac0f8b3c admin admin] PATH_INFO: `/v3/projects/cfa78ac31e2d4894a8855c57206e67ce/users/9c78775627eb48ee9ef35b0b43d5643a/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.118036 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8aaec-0bc0-4e1d-b46f-e794ac0f8b3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfa78ac31e2d4894a8855c57206e67ce, user_id=9c78775627eb48ee9ef35b0b43d5643a, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:06.119495 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: WARNING py.warnings [None req-88b8aaec-0bc0-4e1d-b46f-e794ac0f8b3c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:59:06.119495 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: warnings.warn(msg) Jun 16 18:59:06.119495 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: Jun 16 18:59:06.121421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8aaec-0bc0-4e1d-b46f-e794ac0f8b3c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:06.129679 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5e525f5a-4ec0-4890-9105-1765744e8a9a None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:06.131111 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5e525f5a-4ec0-4890-9105-1765744e8a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:06.139321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 779/1552] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:06 2022] PUT /identity/v3/projects/cfa78ac31e2d4894a8855c57206e67ce/users/9c78775627eb48ee9ef35b0b43d5643a/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:06.140880 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602c1bc-3ceb-4200-a5e9-17b698dc2302 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.141170 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602c1bc-3ceb-4200-a5e9-17b698dc2302 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.141450 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602c1bc-3ceb-4200-a5e9-17b698dc2302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.165322 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4602c1bc-3ceb-4200-a5e9-17b698dc2302 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:06.166544 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4602c1bc-3ceb-4200-a5e9-17b698dc2302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:06.210953 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 774/1553] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:06.212706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018da242-444a-4f7b-89f8-ddbebc8446a2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.213354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018da242-444a-4f7b-89f8-ddbebc8446a2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.213745 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018da242-444a-4f7b-89f8-ddbebc8446a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.233858 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-018da242-444a-4f7b-89f8-ddbebc8446a2 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:06.235080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-018da242-444a-4f7b-89f8-ddbebc8446a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:06.237892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 780/1554] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:06.239173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75a397-fca5-4b21-82b0-4f2c6d2e6703 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.239469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75a397-fca5-4b21-82b0-4f2c6d2e6703 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.239765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75a397-fca5-4b21-82b0-4f2c6d2e6703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.244592 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 775/1555] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:06.245342 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018da242-444a-4f7b-89f8-ddbebc8446a2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.256410 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018da242-444a-4f7b-89f8-ddbebc8446a2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:06.261116 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a58c8-db38-4709-9739-ca69302180d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:06.261795 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a58c8-db38-4709-9739-ca69302180d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.262078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a58c8-db38-4709-9739-ca69302180d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.262381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a58c8-db38-4709-9739-ca69302180d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.263106 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a58c8-db38-4709-9739-ca69302180d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:06.265048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a58c8-db38-4709-9739-ca69302180d5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:06.265167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-3b75a397-fca5-4b21-82b0-4f2c6d2e6703 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:06.266285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3b75a397-fca5-4b21-82b0-4f2c6d2e6703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:06.274651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 781/1556] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:06.276144 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774c627-9651-4906-b587-f6078f168986 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.276438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774c627-9651-4906-b587-f6078f168986 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.276713 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774c627-9651-4906-b587-f6078f168986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.292904 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 776/1557] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:06.294193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd0829-fece-4692-ace8-720aab2f5072 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.294488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd0829-fece-4692-ace8-720aab2f5072 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.294809 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd0829-fece-4692-ace8-720aab2f5072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.297581 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c774c627-9651-4906-b587-f6078f168986 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:06.299232 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c774c627-9651-4906-b587-f6078f168986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:06.304948 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:06.323141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-bcfd0829-fece-4692-ace8-720aab2f5072 None None] MFA Rules not processed for user `9c78775627eb48ee9ef35b0b43d5643a`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:06.324387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bcfd0829-fece-4692-ace8-720aab2f5072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:06.370947 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 782/1558] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:06.375175 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af723e-f2ba-48b6-a85e-e2ac0f468c17 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.375175 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af723e-f2ba-48b6-a85e-e2ac0f468c17 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.375175 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af723e-f2ba-48b6-a85e-e2ac0f468c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.392159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-16af723e-f2ba-48b6-a85e-e2ac0f468c17 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:06.394015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-16af723e-f2ba-48b6-a85e-e2ac0f468c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:06.395451 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:06.402355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 783/1559] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:06.403357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af723e-f2ba-48b6-a85e-e2ac0f468c17 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.404434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 777/1560] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:06.405717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80eaa56-ee36-41e2-aa99-09b817b105b2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.406007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80eaa56-ee36-41e2-aa99-09b817b105b2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.406279 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80eaa56-ee36-41e2-aa99-09b817b105b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.416923 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af723e-f2ba-48b6-a85e-e2ac0f468c17 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:06.421612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f191ef6-9af5-43dc-a6c7-ffecc58b265c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:06.422321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f191ef6-9af5-43dc-a6c7-ffecc58b265c admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.422549 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f191ef6-9af5-43dc-a6c7-ffecc58b265c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.422849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f191ef6-9af5-43dc-a6c7-ffecc58b265c admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.424281 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f191ef6-9af5-43dc-a6c7-ffecc58b265c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:59:06.425135 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c80eaa56-ee36-41e2-aa99-09b817b105b2 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:06.425273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f191ef6-9af5-43dc-a6c7-ffecc58b265c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:06.426454 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c80eaa56-ee36-41e2-aa99-09b817b105b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:06.426877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f191ef6-9af5-43dc-a6c7-ffecc58b265c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:06.442433 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 778/1561] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:06.442433 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17305759-6f4b-4b79-87ac-0fa9405ccf97 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.442433 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17305759-6f4b-4b79-87ac-0fa9405ccf97 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.442433 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17305759-6f4b-4b79-87ac-0fa9405ccf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.444182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 784/1562] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:59:06.444182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f191ef6-9af5-43dc-a6c7-ffecc58b265c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.455120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f191ef6-9af5-43dc-a6c7-ffecc58b265c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:06.462064 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5f93a-6ccd-497a-bea1-85c64c07598e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:06.462854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5f93a-6ccd-497a-bea1-85c64c07598e admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.463427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5f93a-6ccd-497a-bea1-85c64c07598e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.463553 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5f93a-6ccd-497a-bea1-85c64c07598e admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.464307 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5f93a-6ccd-497a-bea1-85c64c07598e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:59:06.464714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5f93a-6ccd-497a-bea1-85c64c07598e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:06.465356 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-17305759-6f4b-4b79-87ac-0fa9405ccf97 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:06.466803 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5f93a-6ccd-497a-bea1-85c64c07598e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:06.467245 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-17305759-6f4b-4b79-87ac-0fa9405ccf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:06.483578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 779/1563] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:06.484417 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17305759-6f4b-4b79-87ac-0fa9405ccf97 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.490714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 785/1564] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:59:06.491805 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c57cd3-618b-4561-b7c4-998b458dc305 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.491805 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c57cd3-618b-4561-b7c4-998b458dc305 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.492150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c57cd3-618b-4561-b7c4-998b458dc305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.495791 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17305759-6f4b-4b79-87ac-0fa9405ccf97 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:06.501960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc72577-b2cc-46ff-8392-76bfbd1ecee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:06.502794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc72577-b2cc-46ff-8392-76bfbd1ecee6 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.503221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc72577-b2cc-46ff-8392-76bfbd1ecee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.503466 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc72577-b2cc-46ff-8392-76bfbd1ecee6 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.504293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc72577-b2cc-46ff-8392-76bfbd1ecee6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:06.505336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-acc72577-b2cc-46ff-8392-76bfbd1ecee6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:59:06.505336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jun 16 18:59:06.505336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: Jun 16 18:59:06.507121 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc72577-b2cc-46ff-8392-76bfbd1ecee6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:06.511093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-72c57cd3-618b-4561-b7c4-998b458dc305 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:06.512303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-72c57cd3-618b-4561-b7c4-998b458dc305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:06.522536 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 786/1565] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:06.524074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12002940-e095-4b2b-8fde-0146e528cddd None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.524561 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12002940-e095-4b2b-8fde-0146e528cddd None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.525042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12002940-e095-4b2b-8fde-0146e528cddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.527359 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 780/1566] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/users => generated 283 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:06.528222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc72577-b2cc-46ff-8392-76bfbd1ecee6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.540839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc72577-b2cc-46ff-8392-76bfbd1ecee6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:06.545868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e101dad-7862-477b-a713-df6db11546f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:06.546549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e101dad-7862-477b-a713-df6db11546f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.546864 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e101dad-7862-477b-a713-df6db11546f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.547146 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e101dad-7862-477b-a713-df6db11546f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.547875 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e101dad-7862-477b-a713-df6db11546f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:59:06.548182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e101dad-7862-477b-a713-df6db11546f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:06.549735 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-12002940-e095-4b2b-8fde-0146e528cddd None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:06.550682 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e101dad-7862-477b-a713-df6db11546f3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:06.551582 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-12002940-e095-4b2b-8fde-0146e528cddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:06.561056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 787/1567] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:06.561907 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12002940-e095-4b2b-8fde-0146e528cddd None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.568562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 781/1568] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:59:06.569865 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6e1cd-2947-436c-b030-b9dbfa41aeb2 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.570164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6e1cd-2947-436c-b030-b9dbfa41aeb2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.570441 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6e1cd-2947-436c-b030-b9dbfa41aeb2 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.571714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 782/1569] 10.222.0.70 () {54 vars in 961 bytes} [Thu Jun 16 18:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:59:06.576805 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6e1cd-2947-436c-b030-b9dbfa41aeb2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.580260 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12002940-e095-4b2b-8fde-0146e528cddd None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:06.585169 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244f846-1479-445e-8472-5c2e6a80f303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:06.585822 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244f846-1479-445e-8472-5c2e6a80f303 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.586100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244f846-1479-445e-8472-5c2e6a80f303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.586376 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244f846-1479-445e-8472-5c2e6a80f303 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.587087 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244f846-1479-445e-8472-5c2e6a80f303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:06.588096 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: WARNING py.warnings [None req-2244f846-1479-445e-8472-5c2e6a80f303 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:59:06.588096 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: warnings.warn(msg) Jun 16 18:59:06.588096 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: Jun 16 18:59:06.589282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244f846-1479-445e-8472-5c2e6a80f303 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:06.591241 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6e1cd-2947-436c-b030-b9dbfa41aeb2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:06.596260 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631511c-ed6e-4c27-9dfd-1e2699743451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:06.596959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631511c-ed6e-4c27-9dfd-1e2699743451 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.597247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631511c-ed6e-4c27-9dfd-1e2699743451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.597527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631511c-ed6e-4c27-9dfd-1e2699743451 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.598258 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631511c-ed6e-4c27-9dfd-1e2699743451 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:59:06.598566 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631511c-ed6e-4c27-9dfd-1e2699743451 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:06.600199 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631511c-ed6e-4c27-9dfd-1e2699743451 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:06.613076 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 783/1570] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:59:06.614045 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631511c-ed6e-4c27-9dfd-1e2699743451 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.622311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 788/1571] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:06.623160 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244f846-1479-445e-8472-5c2e6a80f303 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.624224 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631511c-ed6e-4c27-9dfd-1e2699743451 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:06.687801 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780a903-0ee7-49f8-84b0-3f3544f2e31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:06.688470 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780a903-0ee7-49f8-84b0-3f3544f2e31e admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.688751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780a903-0ee7-49f8-84b0-3f3544f2e31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.689080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780a903-0ee7-49f8-84b0-3f3544f2e31e admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.689974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780a903-0ee7-49f8-84b0-3f3544f2e31e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:06.692013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780a903-0ee7-49f8-84b0-3f3544f2e31e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:06.692869 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244f846-1479-445e-8472-5c2e6a80f303 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:06.697578 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949be1c5-314e-4a13-a93b-c257284f8204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:06.698236 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949be1c5-314e-4a13-a93b-c257284f8204 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.698518 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949be1c5-314e-4a13-a93b-c257284f8204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.698826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949be1c5-314e-4a13-a93b-c257284f8204 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.699546 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949be1c5-314e-4a13-a93b-c257284f8204 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:59:06.699852 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949be1c5-314e-4a13-a93b-c257284f8204 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:06.701454 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949be1c5-314e-4a13-a93b-c257284f8204 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:06.714140 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 789/1572] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:59:06.714802 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949be1c5-314e-4a13-a93b-c257284f8204 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.724602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949be1c5-314e-4a13-a93b-c257284f8204 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:06.726932 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 784/1573] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/projects => generated 403 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:06.727754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780a903-0ee7-49f8-84b0-3f3544f2e31e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.760054 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:06.785046 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b3c25-3d76-4bbc-ae33-a7346a023537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:06.785773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b3c25-3d76-4bbc-ae33-a7346a023537 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.786057 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b3c25-3d76-4bbc-ae33-a7346a023537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.786331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b3c25-3d76-4bbc-ae33-a7346a023537 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.787091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b3c25-3d76-4bbc-ae33-a7346a023537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:06.788705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b3c25-3d76-4bbc-ae33-a7346a023537 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:06.790771 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780a903-0ee7-49f8-84b0-3f3544f2e31e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:06.795466 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c99a31-99db-4ec7-b23b-a541b0e0c2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:06.796131 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c99a31-99db-4ec7-b23b-a541b0e0c2eb admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.796422 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c99a31-99db-4ec7-b23b-a541b0e0c2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.796701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c99a31-99db-4ec7-b23b-a541b0e0c2eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.797422 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c99a31-99db-4ec7-b23b-a541b0e0c2eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:59:06.797726 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c99a31-99db-4ec7-b23b-a541b0e0c2eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:06.800686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 790/1574] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:06.800861 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c99a31-99db-4ec7-b23b-a541b0e0c2eb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:06.800971 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b3c25-3d76-4bbc-ae33-a7346a023537 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.819685 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 785/1575] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:59:06.819884 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b3c25-3d76-4bbc-ae33-a7346a023537 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:06.820015 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09db18-5835-4805-8e65-66cc8e6ef631 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.820015 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09db18-5835-4805-8e65-66cc8e6ef631 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.820015 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09db18-5835-4805-8e65-66cc8e6ef631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.820293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a02b7c-9a37-4193-8cdb-7c68fc4f0af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:06.820293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a02b7c-9a37-4193-8cdb-7c68fc4f0af9 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.820293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a02b7c-9a37-4193-8cdb-7c68fc4f0af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.820293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a02b7c-9a37-4193-8cdb-7c68fc4f0af9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.820643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a02b7c-9a37-4193-8cdb-7c68fc4f0af9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:06.821611 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a02b7c-9a37-4193-8cdb-7c68fc4f0af9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:06.832946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1d09db18-5835-4805-8e65-66cc8e6ef631 None None] MFA Rules not processed for user `8342da838fe4462b9de2817ad3ac4ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:06.834167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1d09db18-5835-4805-8e65-66cc8e6ef631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:06.848847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 791/1576] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:06.849809 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a02b7c-9a37-4193-8cdb-7c68fc4f0af9 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.852854 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:06.906172 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 786/1577] 10.222.0.70 () {58 vars in 1058 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:06.907043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09db18-5835-4805-8e65-66cc8e6ef631 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.913608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a02b7c-9a37-4193-8cdb-7c68fc4f0af9 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:06.918029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09db18-5835-4805-8e65-66cc8e6ef631 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:06.918269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255129f-ffbf-4240-b887-e0a2793e2b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:06.918966 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255129f-ffbf-4240-b887-e0a2793e2b0b admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.919353 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255129f-ffbf-4240-b887-e0a2793e2b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.919631 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255129f-ffbf-4240-b887-e0a2793e2b0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.920325 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255129f-ffbf-4240-b887-e0a2793e2b0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:06.922245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255129f-ffbf-4240-b887-e0a2793e2b0b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:06.922971 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f6646-0592-4ec2-838e-487daf844d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:06.924323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f6646-0592-4ec2-838e-487daf844d98 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:06.924643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f6646-0592-4ec2-838e-487daf844d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:06.924998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f6646-0592-4ec2-838e-487daf844d98 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:06.925673 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f6646-0592-4ec2-838e-487daf844d98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:06.927930 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f6646-0592-4ec2-838e-487daf844d98 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:06.949969 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 792/1578] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/projects => generated 391 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:06.951384 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255129f-ffbf-4240-b887-e0a2793e2b0b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.953881 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 787/1579] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/users => generated 283 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:06.955955 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f6646-0592-4ec2-838e-487daf844d98 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:06.972521 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f6646-0592-4ec2-838e-487daf844d98 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.037920 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255129f-ffbf-4240-b887-e0a2793e2b0b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.042713 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c3532-5077-4b01-8adc-28f6ba0ef78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.042912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301a674-db82-41ac-bb3a-6b40f933b42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.043524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c3532-5077-4b01-8adc-28f6ba0ef78d admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.043644 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301a674-db82-41ac-bb3a-6b40f933b42d admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.043863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301a674-db82-41ac-bb3a-6b40f933b42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.043996 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c3532-5077-4b01-8adc-28f6ba0ef78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.044126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301a674-db82-41ac-bb3a-6b40f933b42d admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.044427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c3532-5077-4b01-8adc-28f6ba0ef78d admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.044804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301a674-db82-41ac-bb3a-6b40f933b42d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.045241 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c3532-5077-4b01-8adc-28f6ba0ef78d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.046752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301a674-db82-41ac-bb3a-6b40f933b42d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.047308 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c3532-5077-4b01-8adc-28f6ba0ef78d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.067522 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 788/1580] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/users => generated 302 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:07.070824 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301a674-db82-41ac-bb3a-6b40f933b42d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.077767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301a674-db82-41ac-bb3a-6b40f933b42d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.077937 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 793/1581] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:06 2022] POST /identity/v3/projects => generated 389 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:07.078627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c3532-5077-4b01-8adc-28f6ba0ef78d admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.104305 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:07.140617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3372b-aa43-4107-b9c9-a4aebf735efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.141304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3372b-aa43-4107-b9c9-a4aebf735efb admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.141595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3372b-aa43-4107-b9c9-a4aebf735efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.141874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3372b-aa43-4107-b9c9-a4aebf735efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.143475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3372b-aa43-4107-b9c9-a4aebf735efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.145069 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3372b-aa43-4107-b9c9-a4aebf735efb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.151921 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c3532-5077-4b01-8adc-28f6ba0ef78d admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.156156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 789/1582] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:07.156627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e862d44-4f7c-483c-9bd5-0bdc266a0f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.157135 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3372b-aa43-4107-b9c9-a4aebf735efb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.157393 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e862d44-4f7c-483c-9bd5-0bdc266a0f40 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.157681 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e862d44-4f7c-483c-9bd5-0bdc266a0f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.157995 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e862d44-4f7c-483c-9bd5-0bdc266a0f40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.158994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e862d44-4f7c-483c-9bd5-0bdc266a0f40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.161163 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e862d44-4f7c-483c-9bd5-0bdc266a0f40 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.168664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3372b-aa43-4107-b9c9-a4aebf735efb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.173783 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14987a8-30e4-485a-b459-d6222fe1cac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.174487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14987a8-30e4-485a-b459-d6222fe1cac7 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.174802 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14987a8-30e4-485a-b459-d6222fe1cac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.175074 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14987a8-30e4-485a-b459-d6222fe1cac7 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.175772 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14987a8-30e4-485a-b459-d6222fe1cac7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.177797 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14987a8-30e4-485a-b459-d6222fe1cac7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.187157 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 794/1583] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:07 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:07.189918 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e862d44-4f7c-483c-9bd5-0bdc266a0f40 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.202606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 790/1584] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:07 2022] POST /identity/v3/users => generated 289 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:07.202606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14987a8-30e4-485a-b459-d6222fe1cac7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.207935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:07.211723 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14987a8-30e4-485a-b459-d6222fe1cac7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.268071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e862d44-4f7c-483c-9bd5-0bdc266a0f40 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.270248 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91e343-c9ad-42c2-8fbc-ce40b29781cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.271037 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91e343-c9ad-42c2-8fbc-ce40b29781cf admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.271319 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91e343-c9ad-42c2-8fbc-ce40b29781cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.271593 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91e343-c9ad-42c2-8fbc-ce40b29781cf admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.272278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91e343-c9ad-42c2-8fbc-ce40b29781cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.274028 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2152a-499d-4a25-ab6e-313590add943 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.274186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91e343-c9ad-42c2-8fbc-ce40b29781cf admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.274778 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2152a-499d-4a25-ab6e-313590add943 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.275063 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2152a-499d-4a25-ab6e-313590add943 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.275344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2152a-499d-4a25-ab6e-313590add943 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.295508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 791/1585] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:07 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:07.296255 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91e343-c9ad-42c2-8fbc-ce40b29781cf admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.306754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91e343-c9ad-42c2-8fbc-ce40b29781cf admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.311363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc40b4-710a-4ae5-b002-29ca4be05000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.312038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc40b4-710a-4ae5-b002-29ca4be05000 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.312325 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc40b4-710a-4ae5-b002-29ca4be05000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.312596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc40b4-710a-4ae5-b002-29ca4be05000 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.313305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc40b4-710a-4ae5-b002-29ca4be05000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.314995 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc40b4-710a-4ae5-b002-29ca4be05000 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.325072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 792/1586] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:07.325930 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc40b4-710a-4ae5-b002-29ca4be05000 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.336155 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc40b4-710a-4ae5-b002-29ca4be05000 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.338131 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2152a-499d-4a25-ab6e-313590add943 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.339824 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2152a-499d-4a25-ab6e-313590add943 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.340972 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d7539-156f-431d-ae9d-22a3c4183648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.341627 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d7539-156f-431d-ae9d-22a3c4183648 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.341949 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d7539-156f-431d-ae9d-22a3c4183648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.342344 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d7539-156f-431d-ae9d-22a3c4183648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.343271 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401d7539-156f-431d-ae9d-22a3c4183648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.345565 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401d7539-156f-431d-ae9d-22a3c4183648 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.349571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 795/1587] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:07.350331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2152a-499d-4a25-ab6e-313590add943 service nova] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.360367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 793/1588] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:07.361820 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2152a-499d-4a25-ab6e-313590add943 service nova] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.362663 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d7539-156f-431d-ae9d-22a3c4183648 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.368356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67066f-9d1b-45d3-a2b8-263e9278d1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.369565 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67066f-9d1b-45d3-a2b8-263e9278d1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.369565 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67066f-9d1b-45d3-a2b8-263e9278d1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.369980 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67066f-9d1b-45d3-a2b8-263e9278d1f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.370833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67066f-9d1b-45d3-a2b8-263e9278d1f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.372941 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67066f-9d1b-45d3-a2b8-263e9278d1f6 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.373088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d7539-156f-431d-ae9d-22a3c4183648 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.379005 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff322768-3090-4a13-ab74-6531474551c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.380559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff322768-3090-4a13-ab74-6531474551c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.381047 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff322768-3090-4a13-ab74-6531474551c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.381574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff322768-3090-4a13-ab74-6531474551c6 admin admin] PATH_INFO: `/v3/projects/63fee94df9fe4790a1566e4931ca8cfa/users/f74d24b6ea574a5a93c3f7f38d0b2014/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.396578 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 796/1589] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:07 2022] POST /identity/v3/users => generated 295 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:07.397409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67066f-9d1b-45d3-a2b8-263e9278d1f6 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.402419 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff322768-3090-4a13-ab74-6531474551c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63fee94df9fe4790a1566e4931ca8cfa, user_id=f74d24b6ea574a5a93c3f7f38d0b2014, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.405309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff322768-3090-4a13-ab74-6531474551c6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.408293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67066f-9d1b-45d3-a2b8-263e9278d1f6 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.413900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642db2c2-88bb-4efb-a10b-4454593e6f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.414757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642db2c2-88bb-4efb-a10b-4454593e6f30 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.415097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642db2c2-88bb-4efb-a10b-4454593e6f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.415738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642db2c2-88bb-4efb-a10b-4454593e6f30 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.416279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642db2c2-88bb-4efb-a10b-4454593e6f30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.418354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642db2c2-88bb-4efb-a10b-4454593e6f30 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.420674 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 794/1590] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:07 2022] PUT /identity/v3/projects/63fee94df9fe4790a1566e4931ca8cfa/users/f74d24b6ea574a5a93c3f7f38d0b2014/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:07.424236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff322768-3090-4a13-ab74-6531474551c6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.446628 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 797/1591] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:07 2022] POST /identity/v3/users => generated 288 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:07.446628 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642db2c2-88bb-4efb-a10b-4454593e6f30 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.457053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642db2c2-88bb-4efb-a10b-4454593e6f30 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.462174 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:07.503753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff322768-3090-4a13-ab74-6531474551c6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.508453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc29b0-72ec-4fb3-b686-4e6e5fb6366a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.509193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc29b0-72ec-4fb3-b686-4e6e5fb6366a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.509540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc29b0-72ec-4fb3-b686-4e6e5fb6366a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.509898 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc29b0-72ec-4fb3-b686-4e6e5fb6366a admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.510756 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc29b0-72ec-4fb3-b686-4e6e5fb6366a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.512538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc29b0-72ec-4fb3-b686-4e6e5fb6366a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.527229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 795/1592] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:07.527229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc29b0-72ec-4fb3-b686-4e6e5fb6366a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.530863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdcad6-ed63-48b5-9c3a-82c1024ef187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.531769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdcad6-ed63-48b5-9c3a-82c1024ef187 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.532129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdcad6-ed63-48b5-9c3a-82c1024ef187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.532498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdcad6-ed63-48b5-9c3a-82c1024ef187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.534181 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afdcad6-ed63-48b5-9c3a-82c1024ef187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.536081 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afdcad6-ed63-48b5-9c3a-82c1024ef187 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.538541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc29b0-72ec-4fb3-b686-4e6e5fb6366a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.543377 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554912c-8c52-475f-b1e1-d16ac2360e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.544200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554912c-8c52-475f-b1e1-d16ac2360e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.544549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554912c-8c52-475f-b1e1-d16ac2360e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.544883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554912c-8c52-475f-b1e1-d16ac2360e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.545716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554912c-8c52-475f-b1e1-d16ac2360e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.547422 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554912c-8c52-475f-b1e1-d16ac2360e16 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.548023 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 798/1593] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:07.548981 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdcad6-ed63-48b5-9c3a-82c1024ef187 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.558773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdcad6-ed63-48b5-9c3a-82c1024ef187 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.565324 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 796/1594] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:07.565502 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976317c9-78ec-4832-97cf-b18cf68d9074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.565502 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976317c9-78ec-4832-97cf-b18cf68d9074 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.565502 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976317c9-78ec-4832-97cf-b18cf68d9074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.565502 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976317c9-78ec-4832-97cf-b18cf68d9074 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.566229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976317c9-78ec-4832-97cf-b18cf68d9074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.566804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554912c-8c52-475f-b1e1-d16ac2360e16 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.568120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976317c9-78ec-4832-97cf-b18cf68d9074 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.576470 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554912c-8c52-475f-b1e1-d16ac2360e16 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.583389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 799/1595] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:07.583389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25813a36-e962-4b1d-b9c8-79d9fc7fa9d5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.583389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25813a36-e962-4b1d-b9c8-79d9fc7fa9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.583389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25813a36-e962-4b1d-b9c8-79d9fc7fa9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.584719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea36f5-5d38-4e60-b351-e92403e36ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.585397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea36f5-5d38-4e60-b351-e92403e36ddd admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.585687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea36f5-5d38-4e60-b351-e92403e36ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.585962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea36f5-5d38-4e60-b351-e92403e36ddd admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.586798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea36f5-5d38-4e60-b351-e92403e36ddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.588398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea36f5-5d38-4e60-b351-e92403e36ddd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.602743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 797/1596] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:07.602743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea36f5-5d38-4e60-b351-e92403e36ddd admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.610194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea36f5-5d38-4e60-b351-e92403e36ddd admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.614668 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-25813a36-e962-4b1d-b9c8-79d9fc7fa9d5 None None] MFA Rules not processed for user `f74d24b6ea574a5a93c3f7f38d0b2014`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:07.615899 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-25813a36-e962-4b1d-b9c8-79d9fc7fa9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:07.618382 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2f46e-ecd8-4abd-a698-dfe6c7579394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.619177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2f46e-ecd8-4abd-a698-dfe6c7579394 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.619540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2f46e-ecd8-4abd-a698-dfe6c7579394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.619901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2f46e-ecd8-4abd-a698-dfe6c7579394 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.620725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2f46e-ecd8-4abd-a698-dfe6c7579394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.622379 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2f46e-ecd8-4abd-a698-dfe6c7579394 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.634835 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 798/1597] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:07.635915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2f46e-ecd8-4abd-a698-dfe6c7579394 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.647649 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2f46e-ecd8-4abd-a698-dfe6c7579394 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.652080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf0a26-6352-47a1-a435-c7801cdd6437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.652817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf0a26-6352-47a1-a435-c7801cdd6437 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.653290 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf0a26-6352-47a1-a435-c7801cdd6437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.653644 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf0a26-6352-47a1-a435-c7801cdd6437 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.654743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bf0a26-6352-47a1-a435-c7801cdd6437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.656576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bf0a26-6352-47a1-a435-c7801cdd6437 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.665549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:07.678020 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 799/1598] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:07.678020 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf0a26-6352-47a1-a435-c7801cdd6437 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.697501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 800/1599] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:07.698870 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25813a36-e962-4b1d-b9c8-79d9fc7fa9d5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.704323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf0a26-6352-47a1-a435-c7801cdd6437 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.710872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b802b37-ac57-4458-ab0e-e1bc4e898305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.711546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b802b37-ac57-4458-ab0e-e1bc4e898305 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.711832 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b802b37-ac57-4458-ab0e-e1bc4e898305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.712112 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b802b37-ac57-4458-ab0e-e1bc4e898305 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.712871 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b802b37-ac57-4458-ab0e-e1bc4e898305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.714460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b802b37-ac57-4458-ab0e-e1bc4e898305 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.717716 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25813a36-e962-4b1d-b9c8-79d9fc7fa9d5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.729739 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 800/1600] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:07.729739 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b802b37-ac57-4458-ab0e-e1bc4e898305 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.729991 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461d5ca-e9b6-4059-b8d5-92fad7e463d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.729991 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461d5ca-e9b6-4059-b8d5-92fad7e463d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.729991 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461d5ca-e9b6-4059-b8d5-92fad7e463d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.729991 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461d5ca-e9b6-4059-b8d5-92fad7e463d1 admin admin] PATH_INFO: `/v3/projects/be909f4e75e64d0a862dde797fe3a522/users/2ef3271eed6847f280a5bd1cc8d7109a/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.739665 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b802b37-ac57-4458-ab0e-e1bc4e898305 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.744514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb3931-c370-4aa4-af83-15fe90a3b5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.745415 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb3931-c370-4aa4-af83-15fe90a3b5c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.745700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb3931-c370-4aa4-af83-15fe90a3b5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.745977 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb3931-c370-4aa4-af83-15fe90a3b5c9 admin admin] PATH_INFO: `/v3/projects/83d71db1ae86479a9cc0a91942f79bd8/users/3975efcc2c874079be3674daca4d9b39/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.746552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461d5ca-e9b6-4059-b8d5-92fad7e463d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be909f4e75e64d0a862dde797fe3a522, user_id=2ef3271eed6847f280a5bd1cc8d7109a, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.749290 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461d5ca-e9b6-4059-b8d5-92fad7e463d1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.764872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb3931-c370-4aa4-af83-15fe90a3b5c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83d71db1ae86479a9cc0a91942f79bd8, user_id=3975efcc2c874079be3674daca4d9b39, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.768811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb3931-c370-4aa4-af83-15fe90a3b5c9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.769615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 801/1601] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:07 2022] PUT /identity/v3/projects/be909f4e75e64d0a862dde797fe3a522/users/2ef3271eed6847f280a5bd1cc8d7109a/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:07.778727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461d5ca-e9b6-4059-b8d5-92fad7e463d1 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.784668 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 801/1602] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:07 2022] PUT /identity/v3/projects/83d71db1ae86479a9cc0a91942f79bd8/users/3975efcc2c874079be3674daca4d9b39/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:07.786076 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc017d-9345-4b59-b1ad-9945107cacd3 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.786528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc017d-9345-4b59-b1ad-9945107cacd3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.786861 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc017d-9345-4b59-b1ad-9945107cacd3 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.788157 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 802/1603] 10.222.0.70 () {54 vars in 912 bytes} [Thu Jun 16 18:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:59:07.788856 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc017d-9345-4b59-b1ad-9945107cacd3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.845025 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461d5ca-e9b6-4059-b8d5-92fad7e463d1 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.849634 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:07.859823 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc017d-9345-4b59-b1ad-9945107cacd3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.865078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc8677-4b4b-48a0-bb79-5c3f50a92f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.865743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc8677-4b4b-48a0-bb79-5c3f50a92f52 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.866023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc8677-4b4b-48a0-bb79-5c3f50a92f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.866300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc8677-4b4b-48a0-bb79-5c3f50a92f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.867054 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc8677-4b4b-48a0-bb79-5c3f50a92f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.868641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc8677-4b4b-48a0-bb79-5c3f50a92f52 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.879900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 803/1604] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:07.880704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc8677-4b4b-48a0-bb79-5c3f50a92f52 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.891043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc8677-4b4b-48a0-bb79-5c3f50a92f52 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.896170 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94928898-b5a9-4dde-b32d-505d3093b54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.897087 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94928898-b5a9-4dde-b32d-505d3093b54a admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.897371 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94928898-b5a9-4dde-b32d-505d3093b54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.897650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94928898-b5a9-4dde-b32d-505d3093b54a admin admin] PATH_INFO: `/v3/projects/7de0652519d3447b8d0e4d948610071f/users/e8813b3f6f8c46519fb827cf521bb37b/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.908365 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09ecfd-ac9f-4394-9b0b-37b2ddd9854a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:07.909128 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09ecfd-ac9f-4394-9b0b-37b2ddd9854a admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:07.909407 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09ecfd-ac9f-4394-9b0b-37b2ddd9854a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:07.909668 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09ecfd-ac9f-4394-9b0b-37b2ddd9854a admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:07.910376 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09ecfd-ac9f-4394-9b0b-37b2ddd9854a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.912014 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09ecfd-ac9f-4394-9b0b-37b2ddd9854a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.913448 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94928898-b5a9-4dde-b32d-505d3093b54a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7de0652519d3447b8d0e4d948610071f, user_id=e8813b3f6f8c46519fb827cf521bb37b, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:07.916326 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94928898-b5a9-4dde-b32d-505d3093b54a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:07.921765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 802/1605] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:07.922548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09ecfd-ac9f-4394-9b0b-37b2ddd9854a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.932116 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 804/1606] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:07 2022] PUT /identity/v3/projects/7de0652519d3447b8d0e4d948610071f/users/e8813b3f6f8c46519fb827cf521bb37b/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:07.932913 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94928898-b5a9-4dde-b32d-505d3093b54a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:07.933380 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09ecfd-ac9f-4394-9b0b-37b2ddd9854a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:07.997362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94928898-b5a9-4dde-b32d-505d3093b54a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:08.002156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc0d90-d48a-4d29-8130-c473368aa258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:08.002779 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c75424-5a2a-4953-9371-a36bf75a4e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:08.003075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc0d90-d48a-4d29-8130-c473368aa258 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.003356 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc0d90-d48a-4d29-8130-c473368aa258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.003460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c75424-5a2a-4953-9371-a36bf75a4e36 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.003632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc0d90-d48a-4d29-8130-c473368aa258 admin admin] PATH_INFO: `/v3/projects/83b304be7bd84301b898bb50f4ac7ae4/users/7f703f74c7ff43a291f54103e164ccdf/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.003839 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c75424-5a2a-4953-9371-a36bf75a4e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.004120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c75424-5a2a-4953-9371-a36bf75a4e36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.004808 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c75424-5a2a-4953-9371-a36bf75a4e36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:08.019340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c75424-5a2a-4953-9371-a36bf75a4e36 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:08.020575 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc0d90-d48a-4d29-8130-c473368aa258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83b304be7bd84301b898bb50f4ac7ae4, user_id=7f703f74c7ff43a291f54103e164ccdf, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:08.023465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc0d90-d48a-4d29-8130-c473368aa258 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:08.043482 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 805/1607] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:07 2022] PUT /identity/v3/projects/83b304be7bd84301b898bb50f4ac7ae4/users/7f703f74c7ff43a291f54103e164ccdf/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:08.048015 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783eb501-4550-41ea-ae66-33e090386f08 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.048301 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783eb501-4550-41ea-ae66-33e090386f08 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.048729 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783eb501-4550-41ea-ae66-33e090386f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.054775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 803/1608] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:07 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:08.056238 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae7db8-82a1-479a-9914-d5e6db13a661 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.056527 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae7db8-82a1-479a-9914-d5e6db13a661 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.056805 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae7db8-82a1-479a-9914-d5e6db13a661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.074890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-cfae7db8-82a1-479a-9914-d5e6db13a661 None None] MFA Rules not processed for user `8342da838fe4462b9de2817ad3ac4ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:08.076098 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-cfae7db8-82a1-479a-9914-d5e6db13a661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:08.079412 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-783eb501-4550-41ea-ae66-33e090386f08 None None] MFA Rules not processed for user `2ef3271eed6847f280a5bd1cc8d7109a`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:08.081421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-783eb501-4550-41ea-ae66-33e090386f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:08.092034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:08.148138 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 804/1609] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Jun 16 18:59:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:08.149644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20122d45-6738-4457-8aec-07ff6c4c441e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.149948 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20122d45-6738-4457-8aec-07ff6c4c441e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.150226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20122d45-6738-4457-8aec-07ff6c4c441e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.155205 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 806/1610] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:08.156120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783eb501-4550-41ea-ae66-33e090386f08 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:08.181642 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-20122d45-6738-4457-8aec-07ff6c4c441e None None] MFA Rules not processed for user `3975efcc2c874079be3674daca4d9b39`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:08.182919 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-20122d45-6738-4457-8aec-07ff6c4c441e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:08.192753 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:08.221408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783eb501-4550-41ea-ae66-33e090386f08 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:08.226269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e9450-90f3-4af2-b7c0-5b4c0eb23e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:08.227202 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e9450-90f3-4af2-b7c0-5b4c0eb23e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.227483 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e9450-90f3-4af2-b7c0-5b4c0eb23e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.227760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e9450-90f3-4af2-b7c0-5b4c0eb23e35 admin admin] PATH_INFO: `/v3/projects/d865cf7b3e5c4291a0204f396143083a/users/8742a73e9b4f46deb0b5c9e1756ffe2e/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.242411 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e9450-90f3-4af2-b7c0-5b4c0eb23e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d865cf7b3e5c4291a0204f396143083a, user_id=8742a73e9b4f46deb0b5c9e1756ffe2e, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:08.245167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e9450-90f3-4af2-b7c0-5b4c0eb23e35 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:08.256805 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 805/1611] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:08.258721 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20122d45-6738-4457-8aec-07ff6c4c441e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:08.263536 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 807/1612] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:08 2022] PUT /identity/v3/projects/d865cf7b3e5c4291a0204f396143083a/users/8742a73e9b4f46deb0b5c9e1756ffe2e/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:08.264730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96daf1-a1bb-4269-975f-d8533a694f4b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.265034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96daf1-a1bb-4269-975f-d8533a694f4b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.265336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96daf1-a1bb-4269-975f-d8533a694f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.269816 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20122d45-6738-4457-8aec-07ff6c4c441e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:08.284665 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-fe96daf1-a1bb-4269-975f-d8533a694f4b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:08.286084 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-fe96daf1-a1bb-4269-975f-d8533a694f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:08.330358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333fb12-961b-498f-8502-9f76b6852821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:08.331306 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333fb12-961b-498f-8502-9f76b6852821 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.331613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333fb12-961b-498f-8502-9f76b6852821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.331875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333fb12-961b-498f-8502-9f76b6852821 admin admin] PATH_INFO: `/v3/projects/057aef18c52b49a382450331694c04d6/users/04327f3d7b844ddb88cf4a81145634c4/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.347060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333fb12-961b-498f-8502-9f76b6852821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=057aef18c52b49a382450331694c04d6, user_id=04327f3d7b844ddb88cf4a81145634c4, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:08.349320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 808/1613] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:08.350219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333fb12-961b-498f-8502-9f76b6852821 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:08.350537 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806d2ed-6e96-4c65-be13-d331e8b3bf3c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.350870 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806d2ed-6e96-4c65-be13-d331e8b3bf3c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.351148 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806d2ed-6e96-4c65-be13-d331e8b3bf3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.368933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 806/1614] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:08 2022] PUT /identity/v3/projects/057aef18c52b49a382450331694c04d6/users/04327f3d7b844ddb88cf4a81145634c4/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:08.368933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333fb12-961b-498f-8502-9f76b6852821 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:08.370334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1806d2ed-6e96-4c65-be13-d331e8b3bf3c None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:08.371588 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1806d2ed-6e96-4c65-be13-d331e8b3bf3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:08.404268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:08.430394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333fb12-961b-498f-8502-9f76b6852821 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:08.435128 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0e12c-9c1c-4999-959f-38564f808ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:08.435828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0e12c-9c1c-4999-959f-38564f808ffc admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.435974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 809/1615] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:08.436148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0e12c-9c1c-4999-959f-38564f808ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.436487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0e12c-9c1c-4999-959f-38564f808ffc admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.436765 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806d2ed-6e96-4c65-be13-d331e8b3bf3c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:08.437212 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0e12c-9c1c-4999-959f-38564f808ffc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:08.439742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0e12c-9c1c-4999-959f-38564f808ffc admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:08.446731 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806d2ed-6e96-4c65-be13-d331e8b3bf3c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:08.452795 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e402c8c-e146-4e16-905e-0b6f04fd1ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:08.453510 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e402c8c-e146-4e16-905e-0b6f04fd1ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.453841 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e402c8c-e146-4e16-905e-0b6f04fd1ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.454148 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e402c8c-e146-4e16-905e-0b6f04fd1ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.455540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e402c8c-e146-4e16-905e-0b6f04fd1ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:08.457574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e402c8c-e146-4e16-905e-0b6f04fd1ec5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:08.465132 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 807/1616] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:08 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:08.465132 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0e12c-9c1c-4999-959f-38564f808ffc admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:08.489342 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 810/1617] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:08 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:08.489342 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e402c8c-e146-4e16-905e-0b6f04fd1ec5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:08.490435 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:08.524653 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0e12c-9c1c-4999-959f-38564f808ffc admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:08.553401 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e402c8c-e146-4e16-905e-0b6f04fd1ec5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:08.558038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448915e-2f7e-4615-addc-e1ee0c5b02e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:08.558728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448915e-2f7e-4615-addc-e1ee0c5b02e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.559014 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448915e-2f7e-4615-addc-e1ee0c5b02e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.559291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448915e-2f7e-4615-addc-e1ee0c5b02e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.559971 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448915e-2f7e-4615-addc-e1ee0c5b02e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:08.561869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448915e-2f7e-4615-addc-e1ee0c5b02e6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:08.576440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34b6b4-79f8-499e-b456-878ab1af915d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:08.577099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34b6b4-79f8-499e-b456-878ab1af915d service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.577381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34b6b4-79f8-499e-b456-878ab1af915d service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.577660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34b6b4-79f8-499e-b456-878ab1af915d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.593223 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 811/1618] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:08 2022] POST /identity/v3/projects => generated 405 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:08.593223 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50548ace-60b4-41b9-abbe-8f5f9b810109 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.593223 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50548ace-60b4-41b9-abbe-8f5f9b810109 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.593223 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50548ace-60b4-41b9-abbe-8f5f9b810109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.609252 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-50548ace-60b4-41b9-abbe-8f5f9b810109 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:08.610450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-50548ace-60b4-41b9-abbe-8f5f9b810109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:08.649308 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34b6b4-79f8-499e-b456-878ab1af915d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:08.651115 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34b6b4-79f8-499e-b456-878ab1af915d service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:08.662338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 808/1619] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:08.664807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34b6b4-79f8-499e-b456-878ab1af915d service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:08.671988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 812/1620] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:08.673915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cb2bd-b8fb-4caf-89c4-a66866029fd6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.674312 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cb2bd-b8fb-4caf-89c4-a66866029fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.674714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cb2bd-b8fb-4caf-89c4-a66866029fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.697216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-206cb2bd-b8fb-4caf-89c4-a66866029fd6 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:08.698941 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-206cb2bd-b8fb-4caf-89c4-a66866029fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:08.707838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 813/1621] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:08.708707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206cb2bd-b8fb-4caf-89c4-a66866029fd6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:08.740103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34b6b4-79f8-499e-b456-878ab1af915d service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:08.744586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:08.771200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206cb2bd-b8fb-4caf-89c4-a66866029fd6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:08.774802 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a15b93-f2a2-4140-84a0-ebbb61eea77f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:08.775480 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a15b93-f2a2-4140-84a0-ebbb61eea77f service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.775770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a15b93-f2a2-4140-84a0-ebbb61eea77f service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.776304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a15b93-f2a2-4140-84a0-ebbb61eea77f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.781474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:08.787124 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a15b93-f2a2-4140-84a0-ebbb61eea77f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:08.788580 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a15b93-f2a2-4140-84a0-ebbb61eea77f service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:08.800923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 814/1622] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:08.801765 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a15b93-f2a2-4140-84a0-ebbb61eea77f service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:08.803649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534dc34-6bd1-40e4-b388-7f6625ddcb57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:08.804512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534dc34-6bd1-40e4-b388-7f6625ddcb57 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.804954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534dc34-6bd1-40e4-b388-7f6625ddcb57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.805386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534dc34-6bd1-40e4-b388-7f6625ddcb57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.812212 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a15b93-f2a2-4140-84a0-ebbb61eea77f service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:08.817772 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534dc34-6bd1-40e4-b388-7f6625ddcb57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:08.818421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366ee3c-0cc4-4ef0-b6ee-361e72df10be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:08.819028 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366ee3c-0cc4-4ef0-b6ee-361e72df10be admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.819310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366ee3c-0cc4-4ef0-b6ee-361e72df10be admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.819578 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534dc34-6bd1-40e4-b388-7f6625ddcb57 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:08.819694 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366ee3c-0cc4-4ef0-b6ee-361e72df10be admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.820350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366ee3c-0cc4-4ef0-b6ee-361e72df10be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:08.822075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366ee3c-0cc4-4ef0-b6ee-361e72df10be admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:08.833441 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 809/1623] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:08.834106 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534dc34-6bd1-40e4-b388-7f6625ddcb57 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:08.837457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 815/1624] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:08.837457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366ee3c-0cc4-4ef0-b6ee-361e72df10be admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:08.845908 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534dc34-6bd1-40e4-b388-7f6625ddcb57 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:08.848681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366ee3c-0cc4-4ef0-b6ee-361e72df10be admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:08.854040 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd415c-57ee-4c4d-8e47-0877685cdbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:08.854236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976236e-4713-4957-8723-7699d5273cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:08.854956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976236e-4713-4957-8723-7699d5273cbb admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.855319 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976236e-4713-4957-8723-7699d5273cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.855630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976236e-4713-4957-8723-7699d5273cbb admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.855867 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd415c-57ee-4c4d-8e47-0877685cdbbd admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.856315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd415c-57ee-4c4d-8e47-0877685cdbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.856441 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7976236e-4713-4957-8723-7699d5273cbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:08.856860 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd415c-57ee-4c4d-8e47-0877685cdbbd admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.857759 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd415c-57ee-4c4d-8e47-0877685cdbbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:08.858606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7976236e-4713-4957-8723-7699d5273cbb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:08.862213 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd415c-57ee-4c4d-8e47-0877685cdbbd admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:08.893227 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 816/1625] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:08 2022] POST /identity/v3/users => generated 309 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:08.893227 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976236e-4713-4957-8723-7699d5273cbb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:08.894505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 810/1626] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:08 2022] POST /identity/v3/users => generated 289 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:08.896032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd415c-57ee-4c4d-8e47-0877685cdbbd admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:08.912499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd415c-57ee-4c4d-8e47-0877685cdbbd admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:08.915899 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976236e-4713-4957-8723-7699d5273cbb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:08.918917 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56939742-2476-44ca-ac31-0022ad4cc1b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:08.920107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56939742-2476-44ca-ac31-0022ad4cc1b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.920800 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56939742-2476-44ca-ac31-0022ad4cc1b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.921281 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56939742-2476-44ca-ac31-0022ad4cc1b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.928707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba475409-855c-4062-b8b8-d5998ebe516b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:08.930793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba475409-855c-4062-b8b8-d5998ebe516b service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.931514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba475409-855c-4062-b8b8-d5998ebe516b service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.932235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba475409-855c-4062-b8b8-d5998ebe516b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:08.939198 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56939742-2476-44ca-ac31-0022ad4cc1b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:08.941957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56939742-2476-44ca-ac31-0022ad4cc1b8 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:08.961112 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba475409-855c-4062-b8b8-d5998ebe516b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:08.963694 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 811/1627] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:08.964300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba475409-855c-4062-b8b8-d5998ebe516b service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:08.964445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56939742-2476-44ca-ac31-0022ad4cc1b8 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:08.984281 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56939742-2476-44ca-ac31-0022ad4cc1b8 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:08.988978 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 817/1628] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:08.991438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba475409-855c-4062-b8b8-d5998ebe516b service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:08.992876 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b094b81-ce3a-4c42-a071-964ce0c522d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:08.994148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b094b81-ce3a-4c42-a071-964ce0c522d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:08.994712 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b094b81-ce3a-4c42-a071-964ce0c522d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:08.995154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b094b81-ce3a-4c42-a071-964ce0c522d5 admin admin] PATH_INFO: `/v3/projects/f85700cbda6d4f34af4c25327b59cadd/users/6e39940406d5488dacb05d95fc28affa/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:09.016208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba475409-855c-4062-b8b8-d5998ebe516b service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:09.019793 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b094b81-ce3a-4c42-a071-964ce0c522d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85700cbda6d4f34af4c25327b59cadd, user_id=6e39940406d5488dacb05d95fc28affa, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:09.024090 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b094b81-ce3a-4c42-a071-964ce0c522d5 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:09.029546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca527f4-9a4c-4a11-9e9d-36cb1f94e911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:09.030355 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca527f4-9a4c-4a11-9e9d-36cb1f94e911 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:09.030687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca527f4-9a4c-4a11-9e9d-36cb1f94e911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:09.030980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca527f4-9a4c-4a11-9e9d-36cb1f94e911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:09.031771 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca527f4-9a4c-4a11-9e9d-36cb1f94e911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:09.034144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca527f4-9a4c-4a11-9e9d-36cb1f94e911 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:09.067779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 818/1629] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:09.068983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca527f4-9a4c-4a11-9e9d-36cb1f94e911 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:09.074334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 812/1630] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:08 2022] PUT /identity/v3/projects/f85700cbda6d4f34af4c25327b59cadd/users/6e39940406d5488dacb05d95fc28affa/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:09.076382 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b094b81-ce3a-4c42-a071-964ce0c522d5 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:09.094330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca527f4-9a4c-4a11-9e9d-36cb1f94e911 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:09.180825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b094b81-ce3a-4c42-a071-964ce0c522d5 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:09.188098 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48f04e-6989-4aa0-8102-0d653a764c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:09.190158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48f04e-6989-4aa0-8102-0d653a764c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:09.190877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48f04e-6989-4aa0-8102-0d653a764c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:09.191167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48f04e-6989-4aa0-8102-0d653a764c81 admin admin] PATH_INFO: `/v3/projects/a67a4534ebbd4d859ea4a0cd62761cc7/users/4516ee81b2d64072b5be7dcd5242f9ac/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:09.223189 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48f04e-6989-4aa0-8102-0d653a764c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a67a4534ebbd4d859ea4a0cd62761cc7, user_id=4516ee81b2d64072b5be7dcd5242f9ac, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:09.225975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48f04e-6989-4aa0-8102-0d653a764c81 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:09.243744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 813/1631] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:09 2022] PUT /identity/v3/projects/a67a4534ebbd4d859ea4a0cd62761cc7/users/4516ee81b2d64072b5be7dcd5242f9ac/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:09.244498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48f04e-6989-4aa0-8102-0d653a764c81 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:09.247147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91dc22-f71e-434a-95c8-7c6427b374da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:09.247956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91dc22-f71e-434a-95c8-7c6427b374da admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:09.249247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91dc22-f71e-434a-95c8-7c6427b374da admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:09.249636 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91dc22-f71e-434a-95c8-7c6427b374da admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:09.255248 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91dc22-f71e-434a-95c8-7c6427b374da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:09.256879 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91dc22-f71e-434a-95c8-7c6427b374da admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:09.284675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 819/1632] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:09.285667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91dc22-f71e-434a-95c8-7c6427b374da admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:09.313301 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91dc22-f71e-434a-95c8-7c6427b374da admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:09.328970 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391a76f-802b-48e2-b195-2c5f2377ceaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:09.329161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48f04e-6989-4aa0-8102-0d653a764c81 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:09.330686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391a76f-802b-48e2-b195-2c5f2377ceaa admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:09.330889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391a76f-802b-48e2-b195-2c5f2377ceaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:09.331150 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391a76f-802b-48e2-b195-2c5f2377ceaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:09.331872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391a76f-802b-48e2-b195-2c5f2377ceaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:09.333483 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391a76f-802b-48e2-b195-2c5f2377ceaa admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:09.335801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0e57e-5c61-4935-86aa-33fc1555e747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:09.336770 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0e57e-5c61-4935-86aa-33fc1555e747 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:09.337120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0e57e-5c61-4935-86aa-33fc1555e747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:09.337624 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0e57e-5c61-4935-86aa-33fc1555e747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:09.338866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0e57e-5c61-4935-86aa-33fc1555e747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:09.341084 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0e57e-5c61-4935-86aa-33fc1555e747 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:09.362707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 820/1633] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:09.363620 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391a76f-802b-48e2-b195-2c5f2377ceaa admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:09.370470 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 814/1634] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:09.371476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0e57e-5c61-4935-86aa-33fc1555e747 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:09.379559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:09.387843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0e57e-5c61-4935-86aa-33fc1555e747 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:09.393374 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391a76f-802b-48e2-b195-2c5f2377ceaa admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:09.398301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4024b6-66e1-49f5-96c5-01e56ea8e898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:09.399431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4024b6-66e1-49f5-96c5-01e56ea8e898 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:09.399878 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4024b6-66e1-49f5-96c5-01e56ea8e898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:09.401255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4024b6-66e1-49f5-96c5-01e56ea8e898 admin admin] PATH_INFO: `/v3/projects/a67a4534ebbd4d859ea4a0cd62761cc7/users/4516ee81b2d64072b5be7dcd5242f9ac/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:09.404412 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acdd02-c16a-49e9-8ffc-8ecad3848bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:09.405055 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4024b6-66e1-49f5-96c5-01e56ea8e898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a67a4534ebbd4d859ea4a0cd62761cc7, user_id=4516ee81b2d64072b5be7dcd5242f9ac, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:09.405319 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acdd02-c16a-49e9-8ffc-8ecad3848bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:09.405621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acdd02-c16a-49e9-8ffc-8ecad3848bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:09.406545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acdd02-c16a-49e9-8ffc-8ecad3848bc0 admin admin] PATH_INFO: `/v3/projects/0da81dfc8a0c4589bc573852844bf522/users/29ed912abba1423f8ea97f5ad71d8777/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:09.410448 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4024b6-66e1-49f5-96c5-01e56ea8e898 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:09.426129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 815/1635] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:09 2022] PUT /identity/v3/projects/a67a4534ebbd4d859ea4a0cd62761cc7/users/4516ee81b2d64072b5be7dcd5242f9ac/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:09.427480 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4024b6-66e1-49f5-96c5-01e56ea8e898 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:09.443107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acdd02-c16a-49e9-8ffc-8ecad3848bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0da81dfc8a0c4589bc573852844bf522, user_id=29ed912abba1423f8ea97f5ad71d8777, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:09.445789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acdd02-c16a-49e9-8ffc-8ecad3848bc0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:09.470760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 821/1636] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:09 2022] PUT /identity/v3/projects/0da81dfc8a0c4589bc573852844bf522/users/29ed912abba1423f8ea97f5ad71d8777/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:09.472670 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbd3b4-5bdf-4ffa-b9d7-7f805fcd3c65 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:09.472971 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbd3b4-5bdf-4ffa-b9d7-7f805fcd3c65 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:09.473258 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbd3b4-5bdf-4ffa-b9d7-7f805fcd3c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:09.552450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c1cbd3b4-5bdf-4ffa-b9d7-7f805fcd3c65 None None] MFA Rules not processed for user `4516ee81b2d64072b5be7dcd5242f9ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:09.553710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c1cbd3b4-5bdf-4ffa-b9d7-7f805fcd3c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:09.588737 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4024b6-66e1-49f5-96c5-01e56ea8e898 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:09.632439 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:09.709897 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 822/1637] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:09 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:09.709897 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbd3b4-5bdf-4ffa-b9d7-7f805fcd3c65 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:09.752254 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7e22b-931d-46e6-af96-a5b4c2b96fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:09.753255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7e22b-931d-46e6-af96-a5b4c2b96fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:09.753540 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7e22b-931d-46e6-af96-a5b4c2b96fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:09.753875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7e22b-931d-46e6-af96-a5b4c2b96fc7 admin admin] PATH_INFO: `/v3/projects/f85700cbda6d4f34af4c25327b59cadd/users/6e39940406d5488dacb05d95fc28affa/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:09.771555 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece7e22b-931d-46e6-af96-a5b4c2b96fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85700cbda6d4f34af4c25327b59cadd, user_id=6e39940406d5488dacb05d95fc28affa, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:09.774901 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece7e22b-931d-46e6-af96-a5b4c2b96fc7 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:09.792600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 816/1638] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:09 2022] PUT /identity/v3/projects/f85700cbda6d4f34af4c25327b59cadd/users/6e39940406d5488dacb05d95fc28affa/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:09.793465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7e22b-931d-46e6-af96-a5b4c2b96fc7 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:09.839209 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbd3b4-5bdf-4ffa-b9d7-7f805fcd3c65 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:09.914695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7e22b-931d-46e6-af96-a5b4c2b96fc7 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:09.926409 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9c412-9960-4455-b5a6-a03d8b88cda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:09.926787 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcfc94-2e65-48fa-9f23-d625817089cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:09.927099 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9c412-9960-4455-b5a6-a03d8b88cda2 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:09.927210 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcfc94-2e65-48fa-9f23-d625817089cb service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:09.927379 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9c412-9960-4455-b5a6-a03d8b88cda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:09.927483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcfc94-2e65-48fa-9f23-d625817089cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:09.927719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcfc94-2e65-48fa-9f23-d625817089cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:09.927844 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9c412-9960-4455-b5a6-a03d8b88cda2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:09.928671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9c412-9960-4455-b5a6-a03d8b88cda2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:09.930843 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9c412-9960-4455-b5a6-a03d8b88cda2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:09.973376 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 823/1639] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:09.979369 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91605c4b-7499-4acc-b6c0-36b72a0bd07f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:09.979969 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91605c4b-7499-4acc-b6c0-36b72a0bd07f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:09.980513 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91605c4b-7499-4acc-b6c0-36b72a0bd07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:10.020719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-91605c4b-7499-4acc-b6c0-36b72a0bd07f None None] MFA Rules not processed for user `6e39940406d5488dacb05d95fc28affa`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:10.021688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:10.024531 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-91605c4b-7499-4acc-b6c0-36b72a0bd07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:10.060847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcfc94-2e65-48fa-9f23-d625817089cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:10.064058 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcfc94-2e65-48fa-9f23-d625817089cb service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:10.083935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 817/1640] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:09 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:10.085629 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcfc94-2e65-48fa-9f23-d625817089cb service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:10.112591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:10.114302 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcfc94-2e65-48fa-9f23-d625817089cb service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:10.121954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87980e6-23e2-4dde-a625-dcf3f3383cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:10.123516 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87980e6-23e2-4dde-a625-dcf3f3383cda admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:10.123894 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87980e6-23e2-4dde-a625-dcf3f3383cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:10.124271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87980e6-23e2-4dde-a625-dcf3f3383cda admin admin] PATH_INFO: `/v3/projects/0da81dfc8a0c4589bc573852844bf522/users/29ed912abba1423f8ea97f5ad71d8777/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:10.127973 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87980e6-23e2-4dde-a625-dcf3f3383cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=0da81dfc8a0c4589bc573852844bf522, user_id=29ed912abba1423f8ea97f5ad71d8777, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:10.132751 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87980e6-23e2-4dde-a625-dcf3f3383cda admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:10.133327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 824/1641] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:09 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:10.144107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91605c4b-7499-4acc-b6c0-36b72a0bd07f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:10.149465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 818/1642] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:10 2022] PUT /identity/v3/projects/0da81dfc8a0c4589bc573852844bf522/users/29ed912abba1423f8ea97f5ad71d8777/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:10.163264 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e58ee-234f-4252-ae6d-b6277d4cc585 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:10.163534 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e58ee-234f-4252-ae6d-b6277d4cc585 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:10.163904 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e58ee-234f-4252-ae6d-b6277d4cc585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:10.167287 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91605c4b-7499-4acc-b6c0-36b72a0bd07f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:10.200733 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-566e58ee-234f-4252-ae6d-b6277d4cc585 None None] MFA Rules not processed for user `29ed912abba1423f8ea97f5ad71d8777`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:10.202708 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-566e58ee-234f-4252-ae6d-b6277d4cc585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:10.259232 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660dc4ae-5ad5-43b9-be7a-18780b87d0f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:10.260061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dc4ae-5ad5-43b9-be7a-18780b87d0f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:10.260457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dc4ae-5ad5-43b9-be7a-18780b87d0f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:10.260670 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dc4ae-5ad5-43b9-be7a-18780b87d0f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:10.353231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 819/1643] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:10 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:10.353231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd37543-d87f-44ef-a0c1-35d41dca7df0 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:10.353780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd37543-d87f-44ef-a0c1-35d41dca7df0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:10.354146 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd37543-d87f-44ef-a0c1-35d41dca7df0 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:10.357486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 820/1644] 10.222.0.70 () {54 vars in 961 bytes} [Thu Jun 16 18:59:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:59:10.360276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd37543-d87f-44ef-a0c1-35d41dca7df0 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:10.383758 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd37543-d87f-44ef-a0c1-35d41dca7df0 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:10.387051 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660dc4ae-5ad5-43b9-be7a-18780b87d0f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:10.388571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660dc4ae-5ad5-43b9-be7a-18780b87d0f6 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:10.394397 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750c779-8b8c-4ea2-bf65-d751724107db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:10.395101 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750c779-8b8c-4ea2-bf65-d751724107db service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:10.395386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750c779-8b8c-4ea2-bf65-d751724107db service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:10.395660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750c779-8b8c-4ea2-bf65-d751724107db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:10.412417 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 825/1645] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:10 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:10.415194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af4129-d633-4357-9eb8-82e2610f42db None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:10.415487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af4129-d633-4357-9eb8-82e2610f42db None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:10.415768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af4129-d633-4357-9eb8-82e2610f42db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:10.456899 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-49af4129-d633-4357-9eb8-82e2610f42db None None] MFA Rules not processed for user `8342da838fe4462b9de2817ad3ac4ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:10.459253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-49af4129-d633-4357-9eb8-82e2610f42db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:10.485909 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750c779-8b8c-4ea2-bf65-d751724107db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:10.487649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750c779-8b8c-4ea2-bf65-d751724107db service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:10.508055 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 821/1646] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:10 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:10.514598 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750c779-8b8c-4ea2-bf65-d751724107db service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:10.532341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750c779-8b8c-4ea2-bf65-d751724107db service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:10.545305 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1af3b-ca47-40a9-9f53-f9f3d50d2f61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:10.546320 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1af3b-ca47-40a9-9f53-f9f3d50d2f61 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:10.546616 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1af3b-ca47-40a9-9f53-f9f3d50d2f61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:10.546910 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1af3b-ca47-40a9-9f53-f9f3d50d2f61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:10.569727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1af3b-ca47-40a9-9f53-f9f3d50d2f61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:10.571299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1af3b-ca47-40a9-9f53-f9f3d50d2f61 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:10.598036 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 822/1647] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:10 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:59:10.603205 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 826/1648] 10.222.0.70 () {58 vars in 1058 bytes} [Thu Jun 16 18:59:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:10.610315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85d320-c06d-4689-94d8-62dbad06f29f None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:10.610882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85d320-c06d-4689-94d8-62dbad06f29f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:10.612069 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85d320-c06d-4689-94d8-62dbad06f29f None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:10.614092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 823/1649] 10.222.0.70 () {54 vars in 912 bytes} [Thu Jun 16 18:59:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:59:10.620800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22736b6d-57a6-4e06-9ec5-bd33aa9d4522 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:10.621121 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22736b6d-57a6-4e06-9ec5-bd33aa9d4522 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:10.621610 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22736b6d-57a6-4e06-9ec5-bd33aa9d4522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:10.645072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-22736b6d-57a6-4e06-9ec5-bd33aa9d4522 None None] MFA Rules not processed for user `8342da838fe4462b9de2817ad3ac4ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:10.648126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-22736b6d-57a6-4e06-9ec5-bd33aa9d4522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:10.662614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 827/1650] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Jun 16 18:59:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:10.734663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85d320-c06d-4689-94d8-62dbad06f29f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:10.757357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85d320-c06d-4689-94d8-62dbad06f29f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:10.772177 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56e238-a28c-46b7-8f75-1b7457b74317 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:10.774529 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56e238-a28c-46b7-8f75-1b7457b74317 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:10.776314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56e238-a28c-46b7-8f75-1b7457b74317 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:10.777075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56e238-a28c-46b7-8f75-1b7457b74317 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:10.855946 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:10.901970 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56e238-a28c-46b7-8f75-1b7457b74317 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:10.904077 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56e238-a28c-46b7-8f75-1b7457b74317 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:10.925213 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 824/1651] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:10 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:11.007271 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22736b6d-57a6-4e06-9ec5-bd33aa9d4522 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:11.023994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22736b6d-57a6-4e06-9ec5-bd33aa9d4522 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:11.036051 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc569ae8-9e77-4391-96e0-f5362c15e924 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:11.040407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc569ae8-9e77-4391-96e0-f5362c15e924 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:11.040913 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc569ae8-9e77-4391-96e0-f5362c15e924 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:11.041872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc569ae8-9e77-4391-96e0-f5362c15e924 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:11.072312 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc569ae8-9e77-4391-96e0-f5362c15e924 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:11.075284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc569ae8-9e77-4391-96e0-f5362c15e924 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:11.118920 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 828/1652] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:11.987871 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56e238-a28c-46b7-8f75-1b7457b74317 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:12.144958 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56e238-a28c-46b7-8f75-1b7457b74317 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:12.151817 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882dd47-f5a3-4165-a437-d76dfef3caf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:12.152955 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882dd47-f5a3-4165-a437-d76dfef3caf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:12.153241 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882dd47-f5a3-4165-a437-d76dfef3caf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:12.153604 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882dd47-f5a3-4165-a437-d76dfef3caf2 admin admin] PATH_INFO: `/v3/users/3975efcc2c874079be3674daca4d9b39` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:12.155049 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d882dd47-f5a3-4165-a437-d76dfef3caf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3975efcc2c874079be3674daca4d9b39)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:12.156234 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: WARNING py.warnings [None req-d882dd47-f5a3-4165-a437-d76dfef3caf2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:59:12.156234 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: warnings.warn(msg) Jun 16 18:59:12.156234 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: Jun 16 18:59:12.159357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d882dd47-f5a3-4165-a437-d76dfef3caf2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:12.273709 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d882dd47-f5a3-4165-a437-d76dfef3caf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3975efcc2c874079be3674daca4d9b39'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:12.274609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d882dd47-f5a3-4165-a437-d76dfef3caf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3975efcc2c874079be3674daca4d9b39'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:12.294005 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d882dd47-f5a3-4165-a437-d76dfef3caf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3975efcc2c874079be3674daca4d9b39'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:12.330734 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d882dd47-f5a3-4165-a437-d76dfef3caf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3975efcc2c874079be3674daca4d9b39'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:12.331860 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d882dd47-f5a3-4165-a437-d76dfef3caf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3975efcc2c874079be3674daca4d9b39'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:12.374039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 825/1653] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:11 2022] DELETE /identity/v3/users/3975efcc2c874079be3674daca4d9b39 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:59:12.379978 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa872e31-5a3c-4d41-9222-009daf506552 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:12.380695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa872e31-5a3c-4d41-9222-009daf506552 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:12.380695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa872e31-5a3c-4d41-9222-009daf506552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:12.406077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-fa872e31-5a3c-4d41-9222-009daf506552 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:12.407483 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-fa872e31-5a3c-4d41-9222-009daf506552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:12.429376 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:12.508139 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 829/1654] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:12.531181 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882dd47-f5a3-4165-a437-d76dfef3caf2 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:12.534032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d882dd47-f5a3-4165-a437-d76dfef3caf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:12.647497 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882dd47-f5a3-4165-a437-d76dfef3caf2 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:12.653339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d90745-e3f5-43c7-afb1-17f5a2c91297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:12.654152 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d90745-e3f5-43c7-afb1-17f5a2c91297 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:12.654537 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d90745-e3f5-43c7-afb1-17f5a2c91297 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:12.655553 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d90745-e3f5-43c7-afb1-17f5a2c91297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:12.669088 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d90745-e3f5-43c7-afb1-17f5a2c91297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:12.670767 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d90745-e3f5-43c7-afb1-17f5a2c91297 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:12.676477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:12.684468 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 826/1655] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:13.365160 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa872e31-5a3c-4d41-9222-009daf506552 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:13.368109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-fa872e31-5a3c-4d41-9222-009daf506552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:13.421540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa872e31-5a3c-4d41-9222-009daf506552 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:13.462602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df97904-358b-4308-bb38-c305232ced91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:13.466991 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df97904-358b-4308-bb38-c305232ced91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:13.467613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df97904-358b-4308-bb38-c305232ced91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:13.468037 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df97904-358b-4308-bb38-c305232ced91 admin admin] PATH_INFO: `/v3/users/9c78775627eb48ee9ef35b0b43d5643a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:13.472194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df97904-358b-4308-bb38-c305232ced91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c78775627eb48ee9ef35b0b43d5643a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:13.477965 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df97904-358b-4308-bb38-c305232ced91 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:13.540480 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d90745-e3f5-43c7-afb1-17f5a2c91297 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:13.543433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-30d90745-e3f5-43c7-afb1-17f5a2c91297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:13.606507 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d90745-e3f5-43c7-afb1-17f5a2c91297 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:13.612850 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db237e-6438-40cd-b859-1e6898749eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:13.613798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db237e-6438-40cd-b859-1e6898749eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:13.614090 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db237e-6438-40cd-b859-1e6898749eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:13.614372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db237e-6438-40cd-b859-1e6898749eca admin admin] PATH_INFO: `/v3/projects/83d71db1ae86479a9cc0a91942f79bd8` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:13.619033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db237e-6438-40cd-b859-1e6898749eca admin admin] RBAC: Authorizing `identity:delete_project(project_id=83d71db1ae86479a9cc0a91942f79bd8)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:13.621661 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5df97904-358b-4308-bb38-c305232ced91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c78775627eb48ee9ef35b0b43d5643a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:13.623796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5df97904-358b-4308-bb38-c305232ced91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c78775627eb48ee9ef35b0b43d5643a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:13.625262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5df97904-358b-4308-bb38-c305232ced91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c78775627eb48ee9ef35b0b43d5643a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:13.625414 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db237e-6438-40cd-b859-1e6898749eca admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:13.644628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5df97904-358b-4308-bb38-c305232ced91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c78775627eb48ee9ef35b0b43d5643a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:13.657231 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5df97904-358b-4308-bb38-c305232ced91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c78775627eb48ee9ef35b0b43d5643a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:13.725951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 830/1656] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:13 2022] DELETE /identity/v3/users/9c78775627eb48ee9ef35b0b43d5643a => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:13.733107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e5327-3c02-44e2-b4a3-111734e6c073 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:13.733834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e5327-3c02-44e2-b4a3-111734e6c073 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:13.734635 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e5327-3c02-44e2-b4a3-111734e6c073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:13.784049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5f1e5327-3c02-44e2-b4a3-111734e6c073 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:13.785899 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5f1e5327-3c02-44e2-b4a3-111734e6c073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:13.789062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-11db237e-6438-40cd-b859-1e6898749eca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83d71db1ae86479a9cc0a91942f79bd8', 'request_id': 'req-11db237e-6438-40cd-b859-1e6898749eca'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:13.806020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-11db237e-6438-40cd-b859-1e6898749eca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83d71db1ae86479a9cc0a91942f79bd8', 'request_id': 'req-11db237e-6438-40cd-b859-1e6898749eca'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:13.831456 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-11db237e-6438-40cd-b859-1e6898749eca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83d71db1ae86479a9cc0a91942f79bd8', 'request_id': 'req-11db237e-6438-40cd-b859-1e6898749eca'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:13.871832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-11db237e-6438-40cd-b859-1e6898749eca admin admin] The token cache is being invalidate because project 83d71db1ae86479a9cc0a91942f79bd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:59:13.872194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-11db237e-6438-40cd-b859-1e6898749eca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:13.874451 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-11db237e-6438-40cd-b859-1e6898749eca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:13.880739 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 827/1657] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 18:59:13 2022] DELETE /identity/v3/projects/83d71db1ae86479a9cc0a91942f79bd8 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:59:13.882737 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db237e-6438-40cd-b859-1e6898749eca admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:13.885939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-11db237e-6438-40cd-b859-1e6898749eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:13.983137 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 831/1658] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:14.007222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e5327-3c02-44e2-b4a3-111734e6c073 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:14.009693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5f1e5327-3c02-44e2-b4a3-111734e6c073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:14.168160 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:14.177488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db237e-6438-40cd-b859-1e6898749eca admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:14.199901 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a2d50-c78e-4f9d-9f97-b6e9573a5ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:14.202062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a2d50-c78e-4f9d-9f97-b6e9573a5ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:14.202357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a2d50-c78e-4f9d-9f97-b6e9573a5ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:14.202804 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a2d50-c78e-4f9d-9f97-b6e9573a5ac1 admin admin] PATH_INFO: `/v3/users/29ed912abba1423f8ea97f5ad71d8777` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:14.225012 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a2d50-c78e-4f9d-9f97-b6e9573a5ac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29ed912abba1423f8ea97f5ad71d8777)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:14.242689 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a2d50-c78e-4f9d-9f97-b6e9573a5ac1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:14.285129 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e5327-3c02-44e2-b4a3-111734e6c073 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:14.299705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c35c2-5398-4ee9-9704-bca0ab2e0baa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:14.301167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c35c2-5398-4ee9-9704-bca0ab2e0baa service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:14.301618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c35c2-5398-4ee9-9704-bca0ab2e0baa service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:14.302617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c35c2-5398-4ee9-9704-bca0ab2e0baa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:14.341498 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c35c2-5398-4ee9-9704-bca0ab2e0baa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:14.346659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c35c2-5398-4ee9-9704-bca0ab2e0baa service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:14.364638 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-359a2d50-c78e-4f9d-9f97-b6e9573a5ac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29ed912abba1423f8ea97f5ad71d8777'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:14.365565 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-359a2d50-c78e-4f9d-9f97-b6e9573a5ac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29ed912abba1423f8ea97f5ad71d8777'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:14.395560 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-359a2d50-c78e-4f9d-9f97-b6e9573a5ac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29ed912abba1423f8ea97f5ad71d8777'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:14.402025 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 832/1659] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:14.421680 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-359a2d50-c78e-4f9d-9f97-b6e9573a5ac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29ed912abba1423f8ea97f5ad71d8777'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:14.423965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-359a2d50-c78e-4f9d-9f97-b6e9573a5ac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29ed912abba1423f8ea97f5ad71d8777'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:14.449422 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:14.457534 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 828/1660] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:13 2022] DELETE /identity/v3/users/29ed912abba1423f8ea97f5ad71d8777 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:15.784684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c35c2-5398-4ee9-9704-bca0ab2e0baa service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:15.808675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c35c2-5398-4ee9-9704-bca0ab2e0baa service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:15.894915 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a2d50-c78e-4f9d-9f97-b6e9573a5ac1 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:15.897040 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-359a2d50-c78e-4f9d-9f97-b6e9573a5ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:16.035210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722c33b-40b5-4c6c-99fc-a5c6058e94bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:16.043034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722c33b-40b5-4c6c-99fc-a5c6058e94bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:16.043034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722c33b-40b5-4c6c-99fc-a5c6058e94bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:16.045728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722c33b-40b5-4c6c-99fc-a5c6058e94bb admin admin] PATH_INFO: `/v3/projects/0da81dfc8a0c4589bc573852844bf522` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:16.051948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722c33b-40b5-4c6c-99fc-a5c6058e94bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0da81dfc8a0c4589bc573852844bf522)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:16.053250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a2d50-c78e-4f9d-9f97-b6e9573a5ac1 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:16.054421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-7722c33b-40b5-4c6c-99fc-a5c6058e94bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:59:16.054421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jun 16 18:59:16.054421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: Jun 16 18:59:16.060264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722c33b-40b5-4c6c-99fc-a5c6058e94bb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:16.067603 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5b1ff-9799-4ae2-8631-f223d605b0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:16.070612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5b1ff-9799-4ae2-8631-f223d605b0de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:16.071047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5b1ff-9799-4ae2-8631-f223d605b0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:16.071342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5b1ff-9799-4ae2-8631-f223d605b0de admin admin] PATH_INFO: `/v3/projects/cfa78ac31e2d4894a8855c57206e67ce` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:16.075151 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5b1ff-9799-4ae2-8631-f223d605b0de admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfa78ac31e2d4894a8855c57206e67ce)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:16.077141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5b1ff-9799-4ae2-8631-f223d605b0de admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:16.184944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-37e5b1ff-9799-4ae2-8631-f223d605b0de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfa78ac31e2d4894a8855c57206e67ce', 'request_id': 'req-37e5b1ff-9799-4ae2-8631-f223d605b0de'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:16.203742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-37e5b1ff-9799-4ae2-8631-f223d605b0de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfa78ac31e2d4894a8855c57206e67ce', 'request_id': 'req-37e5b1ff-9799-4ae2-8631-f223d605b0de'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:16.215891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-37e5b1ff-9799-4ae2-8631-f223d605b0de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfa78ac31e2d4894a8855c57206e67ce', 'request_id': 'req-37e5b1ff-9799-4ae2-8631-f223d605b0de'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:16.248107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-37e5b1ff-9799-4ae2-8631-f223d605b0de admin admin] The token cache is being invalidate because project cfa78ac31e2d4894a8855c57206e67ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:59:16.248460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-37e5b1ff-9799-4ae2-8631-f223d605b0de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:16.249166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-37e5b1ff-9799-4ae2-8631-f223d605b0de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:16.250933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 829/1661] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 18:59:15 2022] DELETE /identity/v3/projects/cfa78ac31e2d4894a8855c57206e67ce => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:59:16.256192 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1885469-d585-413a-be55-63806d5e8f56 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:16.256487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1885469-d585-413a-be55-63806d5e8f56 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:16.256772 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1885469-d585-413a-be55-63806d5e8f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:16.286670 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7722c33b-40b5-4c6c-99fc-a5c6058e94bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0da81dfc8a0c4589bc573852844bf522', 'request_id': 'req-7722c33b-40b5-4c6c-99fc-a5c6058e94bb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:16.309781 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7722c33b-40b5-4c6c-99fc-a5c6058e94bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0da81dfc8a0c4589bc573852844bf522', 'request_id': 'req-7722c33b-40b5-4c6c-99fc-a5c6058e94bb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:16.326145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-a1885469-d585-413a-be55-63806d5e8f56 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:16.327490 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a1885469-d585-413a-be55-63806d5e8f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:16.363712 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7722c33b-40b5-4c6c-99fc-a5c6058e94bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0da81dfc8a0c4589bc573852844bf522', 'request_id': 'req-7722c33b-40b5-4c6c-99fc-a5c6058e94bb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:16.415243 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7722c33b-40b5-4c6c-99fc-a5c6058e94bb admin admin] The token cache is being invalidate because project 0da81dfc8a0c4589bc573852844bf522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:59:16.418168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7722c33b-40b5-4c6c-99fc-a5c6058e94bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:16.430447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7722c33b-40b5-4c6c-99fc-a5c6058e94bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:16.450436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 833/1662] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 18:59:15 2022] DELETE /identity/v3/projects/0da81dfc8a0c4589bc573852844bf522 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:16.472233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9cb55-0c68-4bf5-bef0-f603fc2db79e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:16.473399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9cb55-0c68-4bf5-bef0-f603fc2db79e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:16.475145 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9cb55-0c68-4bf5-bef0-f603fc2db79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:16.554348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 830/1663] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:16.561175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4db9cb55-0c68-4bf5-bef0-f603fc2db79e None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:16.561694 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fd936-22e2-4914-afe1-9d8c1c7abf78 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:16.562055 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fd936-22e2-4914-afe1-9d8c1c7abf78 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:16.562402 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fd936-22e2-4914-afe1-9d8c1c7abf78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:16.565446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4db9cb55-0c68-4bf5-bef0-f603fc2db79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:16.607798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c78fd936-22e2-4914-afe1-9d8c1c7abf78 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:16.607994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 834/1664] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:16.608663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c78fd936-22e2-4914-afe1-9d8c1c7abf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:16.610257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e462e-f4a7-4036-94a6-4264b7aa6bba None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:16.610555 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e462e-f4a7-4036-94a6-4264b7aa6bba None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:16.611506 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e462e-f4a7-4036-94a6-4264b7aa6bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:16.636259 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 831/1665] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:16.640013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78fd936-22e2-4914-afe1-9d8c1c7abf78 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:16.672983 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78fd936-22e2-4914-afe1-9d8c1c7abf78 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:16.690830 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e849a-b6a9-4546-849a-dca5c515fbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:16.691539 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e849a-b6a9-4546-849a-dca5c515fbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:16.691782 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e849a-b6a9-4546-849a-dca5c515fbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:16.692245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e849a-b6a9-4546-849a-dca5c515fbe3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:16.696720 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e849a-b6a9-4546-849a-dca5c515fbe3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:59:16.697046 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e849a-b6a9-4546-849a-dca5c515fbe3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:16.698704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e849a-b6a9-4546-849a-dca5c515fbe3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:16.699505 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-320e462e-f4a7-4036-94a6-4264b7aa6bba None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:16.700824 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-320e462e-f4a7-4036-94a6-4264b7aa6bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:16.744727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 832/1666] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:59:16.744727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e849a-b6a9-4546-849a-dca5c515fbe3 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:16.745072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 835/1667] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:16.745384 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e462e-f4a7-4036-94a6-4264b7aa6bba None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:16.749749 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-da1e849a-b6a9-4546-849a-dca5c515fbe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:16.793923 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e849a-b6a9-4546-849a-dca5c515fbe3 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:16.805575 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e462e-f4a7-4036-94a6-4264b7aa6bba None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:16.808921 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222ad8c-c750-4540-ad60-10734e8d75c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:16.809472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222ad8c-c750-4540-ad60-10734e8d75c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:16.810251 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222ad8c-c750-4540-ad60-10734e8d75c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:16.810251 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222ad8c-c750-4540-ad60-10734e8d75c3 admin admin] PATH_INFO: `/v3/users/f74d24b6ea574a5a93c3f7f38d0b2014` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:16.811322 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222ad8c-c750-4540-ad60-10734e8d75c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f74d24b6ea574a5a93c3f7f38d0b2014)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:16.813863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222ad8c-c750-4540-ad60-10734e8d75c3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:16.830249 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:16.832504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d50ad3-87a8-4bbb-9ba4-51de589c707b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:16.833532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d50ad3-87a8-4bbb-9ba4-51de589c707b admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:16.836770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d50ad3-87a8-4bbb-9ba4-51de589c707b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:16.843495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d50ad3-87a8-4bbb-9ba4-51de589c707b admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:16.845384 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d50ad3-87a8-4bbb-9ba4-51de589c707b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:16.847800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d50ad3-87a8-4bbb-9ba4-51de589c707b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:16.925468 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 836/1668] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:16 2022] POST /identity/v3/projects => generated 367 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:16.926775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d50ad3-87a8-4bbb-9ba4-51de589c707b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:16.947085 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a222ad8c-c750-4540-ad60-10734e8d75c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f74d24b6ea574a5a93c3f7f38d0b2014'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:16.948217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a222ad8c-c750-4540-ad60-10734e8d75c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f74d24b6ea574a5a93c3f7f38d0b2014'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:16.987158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a222ad8c-c750-4540-ad60-10734e8d75c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f74d24b6ea574a5a93c3f7f38d0b2014'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:17.035539 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a222ad8c-c750-4540-ad60-10734e8d75c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f74d24b6ea574a5a93c3f7f38d0b2014'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:17.037074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a222ad8c-c750-4540-ad60-10734e8d75c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f74d24b6ea574a5a93c3f7f38d0b2014'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:17.044673 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:17.073685 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 833/1669] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:16 2022] DELETE /identity/v3/users/f74d24b6ea574a5a93c3f7f38d0b2014 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:17.083809 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222ad8c-c750-4540-ad60-10734e8d75c3 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:17.089039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a222ad8c-c750-4540-ad60-10734e8d75c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:17.192685 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d50ad3-87a8-4bbb-9ba4-51de589c707b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:17.357485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222ad8c-c750-4540-ad60-10734e8d75c3 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:17.366862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c460d4f-2cf9-4058-a473-41db38250213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:17.367758 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c460d4f-2cf9-4058-a473-41db38250213 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:17.368090 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c460d4f-2cf9-4058-a473-41db38250213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:17.368386 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c460d4f-2cf9-4058-a473-41db38250213 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:17.368484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bceb7ad-a28f-4b4d-b9f4-9127412ef88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:17.369173 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c460d4f-2cf9-4058-a473-41db38250213 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:59:17.369271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceb7ad-a28f-4b4d-b9f4-9127412ef88d admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:17.369496 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c460d4f-2cf9-4058-a473-41db38250213 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:17.370460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceb7ad-a28f-4b4d-b9f4-9127412ef88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:17.371174 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c460d4f-2cf9-4058-a473-41db38250213 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:17.371548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceb7ad-a28f-4b4d-b9f4-9127412ef88d admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:17.373008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bceb7ad-a28f-4b4d-b9f4-9127412ef88d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:17.380254 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bceb7ad-a28f-4b4d-b9f4-9127412ef88d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:17.394394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 837/1670] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 468 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:59:17.395892 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634485d-6636-4069-9e8b-06d648fa4b07 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:17.396198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634485d-6636-4069-9e8b-06d648fa4b07 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:17.396488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634485d-6636-4069-9e8b-06d648fa4b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:17.426101 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 834/1671] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:17 2022] POST /identity/v3/users => generated 284 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:17.427092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bceb7ad-a28f-4b4d-b9f4-9127412ef88d admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:17.428647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2634485d-6636-4069-9e8b-06d648fa4b07 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:17.430938 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2634485d-6636-4069-9e8b-06d648fa4b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:17.433452 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7bceb7ad-a28f-4b4d-b9f4-9127412ef88d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:17.443244 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 838/1672] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:17.443244 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634485d-6636-4069-9e8b-06d648fa4b07 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:17.452605 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634485d-6636-4069-9e8b-06d648fa4b07 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:17.464071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bceb7ad-a28f-4b4d-b9f4-9127412ef88d admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:17.464277 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32fbac-01b9-47b8-9d0e-e9dbb39be9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:17.465025 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32fbac-01b9-47b8-9d0e-e9dbb39be9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:17.465318 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32fbac-01b9-47b8-9d0e-e9dbb39be9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:17.465601 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32fbac-01b9-47b8-9d0e-e9dbb39be9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:17.467531 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32fbac-01b9-47b8-9d0e-e9dbb39be9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:17.471812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32fbac-01b9-47b8-9d0e-e9dbb39be9d7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:17.473653 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed760f-11ab-4f0e-931c-244f986c1fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:17.476899 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed760f-11ab-4f0e-931c-244f986c1fec admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:17.477398 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed760f-11ab-4f0e-931c-244f986c1fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:17.477756 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed760f-11ab-4f0e-931c-244f986c1fec admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:17.478538 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed760f-11ab-4f0e-931c-244f986c1fec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:17.482666 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed760f-11ab-4f0e-931c-244f986c1fec admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:17.499630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 839/1673] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:17.501041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32fbac-01b9-47b8-9d0e-e9dbb39be9d7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:17.509698 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bc32fbac-01b9-47b8-9d0e-e9dbb39be9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:17.549110 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 835/1674] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:17 2022] POST /identity/v3/projects => generated 357 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:17.550053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed760f-11ab-4f0e-931c-244f986c1fec admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:17.659001 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32fbac-01b9-47b8-9d0e-e9dbb39be9d7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:17.687383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:17.715126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed760f-11ab-4f0e-931c-244f986c1fec admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:17.728762 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103e298-0a80-49a9-86db-16f2ee27a9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:17.729476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103e298-0a80-49a9-86db-16f2ee27a9d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:17.729762 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103e298-0a80-49a9-86db-16f2ee27a9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:17.730039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103e298-0a80-49a9-86db-16f2ee27a9d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:17.730823 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103e298-0a80-49a9-86db-16f2ee27a9d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:17.733711 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103e298-0a80-49a9-86db-16f2ee27a9d9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:17.762019 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 836/1675] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:17.764444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103e298-0a80-49a9-86db-16f2ee27a9d9 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:17.788818 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103e298-0a80-49a9-86db-16f2ee27a9d9 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:17.796422 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b7963-9a02-40a2-b3c8-3e2248cf3282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:17.797196 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b7963-9a02-40a2-b3c8-3e2248cf3282 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:17.798069 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b7963-9a02-40a2-b3c8-3e2248cf3282 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:17.799211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b7963-9a02-40a2-b3c8-3e2248cf3282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:17.803142 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da041e93-1b44-48f5-b5ce-d9e1a0fef109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:17.803939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da041e93-1b44-48f5-b5ce-d9e1a0fef109 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:17.804229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da041e93-1b44-48f5-b5ce-d9e1a0fef109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:17.804507 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da041e93-1b44-48f5-b5ce-d9e1a0fef109 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:17.805285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da041e93-1b44-48f5-b5ce-d9e1a0fef109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:17.808211 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da041e93-1b44-48f5-b5ce-d9e1a0fef109 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:17.825414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b7963-9a02-40a2-b3c8-3e2248cf3282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:17.830264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b7963-9a02-40a2-b3c8-3e2248cf3282 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:17.856616 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 837/1676] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:17 2022] POST /identity/v3/users => generated 279 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:17.856616 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da041e93-1b44-48f5-b5ce-d9e1a0fef109 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:17.856874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 840/1677] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:17.857699 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b7963-9a02-40a2-b3c8-3e2248cf3282 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:17.880964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da041e93-1b44-48f5-b5ce-d9e1a0fef109 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:17.891895 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b7963-9a02-40a2-b3c8-3e2248cf3282 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:17.894380 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f3d37-1d5b-4b5f-879a-5449e663f4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:17.895344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3d37-1d5b-4b5f-879a-5449e663f4f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:17.895633 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3d37-1d5b-4b5f-879a-5449e663f4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:17.895912 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3d37-1d5b-4b5f-879a-5449e663f4f9 admin admin] PATH_INFO: `/v3/projects/df6eed19c14e457984f8bd65673c6134/users/0173edf0b4a44f6e9012194f2cd5f691/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:17.905937 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83bc11-3309-4ee3-b45d-705abbe88572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:17.906853 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83bc11-3309-4ee3-b45d-705abbe88572 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:17.906969 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83bc11-3309-4ee3-b45d-705abbe88572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:17.910657 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83bc11-3309-4ee3-b45d-705abbe88572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:17.911107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83bc11-3309-4ee3-b45d-705abbe88572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:17.913354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83bc11-3309-4ee3-b45d-705abbe88572 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:17.922789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:17.939234 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 841/1678] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:17.940263 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83bc11-3309-4ee3-b45d-705abbe88572 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:17.945789 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f3d37-1d5b-4b5f-879a-5449e663f4f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df6eed19c14e457984f8bd65673c6134, user_id=0173edf0b4a44f6e9012194f2cd5f691, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:17.953029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f3d37-1d5b-4b5f-879a-5449e663f4f9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:17.969508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83bc11-3309-4ee3-b45d-705abbe88572 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:17.987904 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 838/1679] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:17 2022] PUT /identity/v3/projects/df6eed19c14e457984f8bd65673c6134/users/0173edf0b4a44f6e9012194f2cd5f691/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:17.998372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72484ea-cc4b-420b-931f-64914593ebde None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:17.998712 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72484ea-cc4b-420b-931f-64914593ebde None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:17.998993 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72484ea-cc4b-420b-931f-64914593ebde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:18.085755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d72484ea-cc4b-420b-931f-64914593ebde None None] MFA Rules not processed for user `0173edf0b4a44f6e9012194f2cd5f691`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:18.096822 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d72484ea-cc4b-420b-931f-64914593ebde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:18.182441 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcba38-6af0-443f-b8a3-44c0884d71f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:18.183343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcba38-6af0-443f-b8a3-44c0884d71f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:18.183727 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcba38-6af0-443f-b8a3-44c0884d71f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:18.184391 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcba38-6af0-443f-b8a3-44c0884d71f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:18.185659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcba38-6af0-443f-b8a3-44c0884d71f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:18.191883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcba38-6af0-443f-b8a3-44c0884d71f1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:18.232902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 842/1680] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:18.241749 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcba38-6af0-443f-b8a3-44c0884d71f1 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:18.275497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcba38-6af0-443f-b8a3-44c0884d71f1 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:18.303638 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583b788-d5ba-4d51-9488-548c25085685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:18.304569 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583b788-d5ba-4d51-9488-548c25085685 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:18.304848 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583b788-d5ba-4d51-9488-548c25085685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:18.305120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583b788-d5ba-4d51-9488-548c25085685 admin admin] PATH_INFO: `/v3/projects/d1d433a6ea254dc8844b9cf736fee3f0/users/c83a0e48174b44f5916004cd83487b34/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:18.355495 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 839/1681] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:18.363312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72484ea-cc4b-420b-931f-64914593ebde None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:18.366807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d72484ea-cc4b-420b-931f-64914593ebde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:18.367906 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583b788-d5ba-4d51-9488-548c25085685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d433a6ea254dc8844b9cf736fee3f0, user_id=c83a0e48174b44f5916004cd83487b34, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:18.375675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583b788-d5ba-4d51-9488-548c25085685 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:18.430629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 843/1682] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:18 2022] PUT /identity/v3/projects/d1d433a6ea254dc8844b9cf736fee3f0/users/c83a0e48174b44f5916004cd83487b34/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:18.432667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a15ed-64e0-4f6b-970d-ab220b62ab26 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:18.432897 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a15ed-64e0-4f6b-970d-ab220b62ab26 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:18.433146 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a15ed-64e0-4f6b-970d-ab220b62ab26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:18.515656 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-de8a15ed-64e0-4f6b-970d-ab220b62ab26 None None] MFA Rules not processed for user `c83a0e48174b44f5916004cd83487b34`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:18.517490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-de8a15ed-64e0-4f6b-970d-ab220b62ab26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:18.554061 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72484ea-cc4b-420b-931f-64914593ebde None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:18.683644 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 844/1683] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:18 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:59:18.684400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a15ed-64e0-4f6b-970d-ab220b62ab26 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:18.699625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74561b-b9e9-4765-8e53-ca32a61922b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:18.700396 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74561b-b9e9-4765-8e53-ca32a61922b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:18.700687 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74561b-b9e9-4765-8e53-ca32a61922b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:18.700967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74561b-b9e9-4765-8e53-ca32a61922b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:18.712278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:18.877450 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74561b-b9e9-4765-8e53-ca32a61922b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:18.883201 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74561b-b9e9-4765-8e53-ca32a61922b6 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:18.896379 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a15ed-64e0-4f6b-970d-ab220b62ab26 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:18.906851 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 840/1684] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:18 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:59:18.911348 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e69c85-89b0-4024-944b-367728a10234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:18.912391 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e69c85-89b0-4024-944b-367728a10234 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:18.913108 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e69c85-89b0-4024-944b-367728a10234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:18.913790 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e69c85-89b0-4024-944b-367728a10234 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:18.914951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e69c85-89b0-4024-944b-367728a10234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:18.919368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e69c85-89b0-4024-944b-367728a10234 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:18.939853 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:18.991488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 845/1685] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:18 2022] POST /identity/v3/projects => generated 357 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:18.996569 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74561b-b9e9-4765-8e53-ca32a61922b6 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:19.018734 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74561b-b9e9-4765-8e53-ca32a61922b6 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:19.209963 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d84cf0-8f90-4ae0-8cd4-afcf4051200e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:19.210587 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d84cf0-8f90-4ae0-8cd4-afcf4051200e admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:19.210880 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d84cf0-8f90-4ae0-8cd4-afcf4051200e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:19.211459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d84cf0-8f90-4ae0-8cd4-afcf4051200e admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:19.211844 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d84cf0-8f90-4ae0-8cd4-afcf4051200e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:19.213791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d84cf0-8f90-4ae0-8cd4-afcf4051200e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:19.269218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 841/1686] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:18 2022] POST /identity/v3/users => generated 285 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:19.270648 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e69c85-89b0-4024-944b-367728a10234 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:19.291703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e69c85-89b0-4024-944b-367728a10234 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:19.307459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8161363-1024-40c6-a31c-3ef920922a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:19.311696 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8161363-1024-40c6-a31c-3ef920922a57 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:19.312146 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8161363-1024-40c6-a31c-3ef920922a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:19.312548 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8161363-1024-40c6-a31c-3ef920922a57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:19.313593 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8161363-1024-40c6-a31c-3ef920922a57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:19.319399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8161363-1024-40c6-a31c-3ef920922a57 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:19.352228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 846/1687] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:19.357663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d84cf0-8f90-4ae0-8cd4-afcf4051200e admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:19.377207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d84cf0-8f90-4ae0-8cd4-afcf4051200e admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:19.397476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bc116-31d3-4536-81fd-f9d004339009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:19.398458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bc116-31d3-4536-81fd-f9d004339009 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:19.398771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bc116-31d3-4536-81fd-f9d004339009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:19.399051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bc116-31d3-4536-81fd-f9d004339009 admin admin] PATH_INFO: `/v3/projects/b4e33caa7ba84f0586fd544b6335e7b8/users/864b638582984f8d82e46901c106e65c/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:19.445095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bc116-31d3-4536-81fd-f9d004339009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e33caa7ba84f0586fd544b6335e7b8, user_id=864b638582984f8d82e46901c106e65c, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:19.449860 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bc116-31d3-4536-81fd-f9d004339009 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:19.487127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 842/1688] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:19 2022] PUT /identity/v3/projects/b4e33caa7ba84f0586fd544b6335e7b8/users/864b638582984f8d82e46901c106e65c/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:19.492224 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8161363-1024-40c6-a31c-3ef920922a57 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:19.510200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8161363-1024-40c6-a31c-3ef920922a57 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:19.708824 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea097608-c347-4d1c-854e-cb2c9325e7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:19.710852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea097608-c347-4d1c-854e-cb2c9325e7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:19.713199 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea097608-c347-4d1c-854e-cb2c9325e7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:19.713986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea097608-c347-4d1c-854e-cb2c9325e7b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:19.715206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea097608-c347-4d1c-854e-cb2c9325e7b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:19.719436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea097608-c347-4d1c-854e-cb2c9325e7b7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:19.759091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 847/1689] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:19.760532 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bc116-31d3-4536-81fd-f9d004339009 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:19.800443 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bc116-31d3-4536-81fd-f9d004339009 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:19.817370 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc55f6-2833-48c3-9b75-0aa04984a09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:19.818757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc55f6-2833-48c3-9b75-0aa04984a09c admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:19.820435 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc55f6-2833-48c3-9b75-0aa04984a09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:19.820943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc55f6-2833-48c3-9b75-0aa04984a09c admin admin] PATH_INFO: `/v3/projects/b4e33caa7ba84f0586fd544b6335e7b8/users/864b638582984f8d82e46901c106e65c/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:19.828237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc55f6-2833-48c3-9b75-0aa04984a09c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e33caa7ba84f0586fd544b6335e7b8, user_id=864b638582984f8d82e46901c106e65c, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:19.837897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc55f6-2833-48c3-9b75-0aa04984a09c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:19.887690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 843/1690] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:19 2022] PUT /identity/v3/projects/b4e33caa7ba84f0586fd544b6335e7b8/users/864b638582984f8d82e46901c106e65c/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:19.899638 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f293bb-c8c7-4610-b681-640502f60178 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:19.900195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f293bb-c8c7-4610-b681-640502f60178 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:19.900195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f293bb-c8c7-4610-b681-640502f60178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:19.970461 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-58f293bb-c8c7-4610-b681-640502f60178 None None] MFA Rules not processed for user `864b638582984f8d82e46901c106e65c`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:19.972828 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-58f293bb-c8c7-4610-b681-640502f60178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:20.182315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 848/1691] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:19 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:20.200779 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc55f6-2833-48c3-9b75-0aa04984a09c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:20.353225 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc55f6-2833-48c3-9b75-0aa04984a09c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:20.369297 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386958e-6f25-4473-82a7-74ea9a2c4c3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:20.370122 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386958e-6f25-4473-82a7-74ea9a2c4c3a service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:20.370484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386958e-6f25-4473-82a7-74ea9a2c4c3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:20.370910 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386958e-6f25-4473-82a7-74ea9a2c4c3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:20.389432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386958e-6f25-4473-82a7-74ea9a2c4c3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:20.393343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386958e-6f25-4473-82a7-74ea9a2c4c3a service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:20.411624 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:20.441163 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f293bb-c8c7-4610-b681-640502f60178 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:20.442910 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 844/1692] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:20 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:59:20.443668 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-58f293bb-c8c7-4610-b681-640502f60178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:20.592229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386958e-6f25-4473-82a7-74ea9a2c4c3a service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:20.631390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386958e-6f25-4473-82a7-74ea9a2c4c3a service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:20.652472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9c2bc-8a3f-4b2a-9b55-2a2fe059fc51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:20.653543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9c2bc-8a3f-4b2a-9b55-2a2fe059fc51 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:20.653978 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9c2bc-8a3f-4b2a-9b55-2a2fe059fc51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:20.654390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9c2bc-8a3f-4b2a-9b55-2a2fe059fc51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:20.676886 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:20.686610 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f293bb-c8c7-4610-b681-640502f60178 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:20.725371 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01c9a1-6901-4be4-852f-05b87c7c681a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:20.727039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c9a1-6901-4be4-852f-05b87c7c681a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:20.727340 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c9a1-6901-4be4-852f-05b87c7c681a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:20.727626 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c9a1-6901-4be4-852f-05b87c7c681a admin admin] PATH_INFO: `/v3/projects/63fee94df9fe4790a1566e4931ca8cfa` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:20.729035 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01c9a1-6901-4be4-852f-05b87c7c681a admin admin] RBAC: Authorizing `identity:delete_project(project_id=63fee94df9fe4790a1566e4931ca8cfa)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:20.731045 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01c9a1-6901-4be4-852f-05b87c7c681a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:20.868795 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3c01c9a1-6901-4be4-852f-05b87c7c681a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63fee94df9fe4790a1566e4931ca8cfa', 'request_id': 'req-3c01c9a1-6901-4be4-852f-05b87c7c681a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:20.873593 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9c2bc-8a3f-4b2a-9b55-2a2fe059fc51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:20.875253 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9c2bc-8a3f-4b2a-9b55-2a2fe059fc51 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:20.887644 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3c01c9a1-6901-4be4-852f-05b87c7c681a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63fee94df9fe4790a1566e4931ca8cfa', 'request_id': 'req-3c01c9a1-6901-4be4-852f-05b87c7c681a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:20.899571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3c01c9a1-6901-4be4-852f-05b87c7c681a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63fee94df9fe4790a1566e4931ca8cfa', 'request_id': 'req-3c01c9a1-6901-4be4-852f-05b87c7c681a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:20.907640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 845/1693] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:20 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:59:20.918410 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3c01c9a1-6901-4be4-852f-05b87c7c681a admin admin] The token cache is being invalidate because project 63fee94df9fe4790a1566e4931ca8cfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:59:20.918845 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3c01c9a1-6901-4be4-852f-05b87c7c681a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:20.920058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3c01c9a1-6901-4be4-852f-05b87c7c681a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:20.922448 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 849/1694] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 18:59:20 2022] DELETE /identity/v3/projects/63fee94df9fe4790a1566e4931ca8cfa => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:20.927896 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9c2bc-8a3f-4b2a-9b55-2a2fe059fc51 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:20.957039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9c2bc-8a3f-4b2a-9b55-2a2fe059fc51 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:21.191614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ba99e-ce57-4f22-94d0-dd0d48171a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:21.192532 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ba99e-ce57-4f22-94d0-dd0d48171a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:21.192816 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ba99e-ce57-4f22-94d0-dd0d48171a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:21.193099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ba99e-ce57-4f22-94d0-dd0d48171a60 admin admin] PATH_INFO: `/v3/users/6e39940406d5488dacb05d95fc28affa` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:21.206231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ba99e-ce57-4f22-94d0-dd0d48171a60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e39940406d5488dacb05d95fc28affa)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:21.206231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ba99e-ce57-4f22-94d0-dd0d48171a60 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:21.402544 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-5f5ba99e-ce57-4f22-94d0-dd0d48171a60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e39940406d5488dacb05d95fc28affa'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:21.407056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-5f5ba99e-ce57-4f22-94d0-dd0d48171a60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e39940406d5488dacb05d95fc28affa'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:21.437471 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-5f5ba99e-ce57-4f22-94d0-dd0d48171a60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e39940406d5488dacb05d95fc28affa'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:21.511693 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-5f5ba99e-ce57-4f22-94d0-dd0d48171a60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e39940406d5488dacb05d95fc28affa'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:21.512484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-5f5ba99e-ce57-4f22-94d0-dd0d48171a60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e39940406d5488dacb05d95fc28affa'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:21.556638 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 846/1695] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:20 2022] DELETE /identity/v3/users/6e39940406d5488dacb05d95fc28affa => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:23.080395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01c9a1-6901-4be4-852f-05b87c7c681a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:23.083747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3c01c9a1-6901-4be4-852f-05b87c7c681a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:23.114957 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01c9a1-6901-4be4-852f-05b87c7c681a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:23.202848 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ba99e-ce57-4f22-94d0-dd0d48171a60 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:23.205257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5f5ba99e-ce57-4f22-94d0-dd0d48171a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:23.273051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:23.280934 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee033e3-f709-4720-b858-531911ad9948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:23.287838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee033e3-f709-4720-b858-531911ad9948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:23.288379 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee033e3-f709-4720-b858-531911ad9948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:23.290664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee033e3-f709-4720-b858-531911ad9948 admin admin] PATH_INFO: `/v3/projects/f85700cbda6d4f34af4c25327b59cadd` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:23.293800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee033e3-f709-4720-b858-531911ad9948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f85700cbda6d4f34af4c25327b59cadd)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:23.299979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee033e3-f709-4720-b858-531911ad9948 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:23.410786 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cee033e3-f709-4720-b858-531911ad9948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f85700cbda6d4f34af4c25327b59cadd', 'request_id': 'req-cee033e3-f709-4720-b858-531911ad9948'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:23.417998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ba99e-ce57-4f22-94d0-dd0d48171a60 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:23.445291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cee033e3-f709-4720-b858-531911ad9948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f85700cbda6d4f34af4c25327b59cadd', 'request_id': 'req-cee033e3-f709-4720-b858-531911ad9948'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:23.465540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cee033e3-f709-4720-b858-531911ad9948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f85700cbda6d4f34af4c25327b59cadd', 'request_id': 'req-cee033e3-f709-4720-b858-531911ad9948'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:23.492220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cee033e3-f709-4720-b858-531911ad9948 admin admin] The token cache is being invalidate because project f85700cbda6d4f34af4c25327b59cadd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:59:23.492897 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cee033e3-f709-4720-b858-531911ad9948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:23.495419 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cee033e3-f709-4720-b858-531911ad9948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:23.498269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 850/1696] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 18:59:23 2022] DELETE /identity/v3/projects/f85700cbda6d4f34af4c25327b59cadd => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:23.506443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f906ceb-68df-4e61-9121-19d41d7d9f20 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:23.506966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f906ceb-68df-4e61-9121-19d41d7d9f20 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:23.507385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f906ceb-68df-4e61-9121-19d41d7d9f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:23.518909 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd47b30-c250-4b0d-b99b-726a7d438803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:23.520076 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd47b30-c250-4b0d-b99b-726a7d438803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:23.520511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd47b30-c250-4b0d-b99b-726a7d438803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:23.520928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd47b30-c250-4b0d-b99b-726a7d438803 admin admin] PATH_INFO: `/v3/users/2ef3271eed6847f280a5bd1cc8d7109a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:23.522804 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd47b30-c250-4b0d-b99b-726a7d438803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef3271eed6847f280a5bd1cc8d7109a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:23.526215 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd47b30-c250-4b0d-b99b-726a7d438803 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:23.552354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1f906ceb-68df-4e61-9121-19d41d7d9f20 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:23.553787 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1f906ceb-68df-4e61-9121-19d41d7d9f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:23.578690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 851/1697] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:23.589985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f2a0e-6824-4de2-9d9d-7265b3111b4b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:23.591400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f2a0e-6824-4de2-9d9d-7265b3111b4b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:23.592710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f2a0e-6824-4de2-9d9d-7265b3111b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:23.630338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3cd47b30-c250-4b0d-b99b-726a7d438803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef3271eed6847f280a5bd1cc8d7109a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:23.631684 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3cd47b30-c250-4b0d-b99b-726a7d438803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef3271eed6847f280a5bd1cc8d7109a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:23.646013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c17f2a0e-6824-4de2-9d9d-7265b3111b4b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:23.652248 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c17f2a0e-6824-4de2-9d9d-7265b3111b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:23.657204 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3cd47b30-c250-4b0d-b99b-726a7d438803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef3271eed6847f280a5bd1cc8d7109a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:23.695902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 852/1698] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:23.697044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f2a0e-6824-4de2-9d9d-7265b3111b4b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:23.707220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:23.710037 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3cd47b30-c250-4b0d-b99b-726a7d438803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef3271eed6847f280a5bd1cc8d7109a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:23.710935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3cd47b30-c250-4b0d-b99b-726a7d438803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef3271eed6847f280a5bd1cc8d7109a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:23.723796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f2a0e-6824-4de2-9d9d-7265b3111b4b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:23.732547 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa0da3-3ee1-4955-bc33-a835a337713b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:23.734000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa0da3-3ee1-4955-bc33-a835a337713b admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:23.738064 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa0da3-3ee1-4955-bc33-a835a337713b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:23.740052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa0da3-3ee1-4955-bc33-a835a337713b admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:23.744025 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 847/1699] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:23 2022] DELETE /identity/v3/users/2ef3271eed6847f280a5bd1cc8d7109a => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:59:23.744221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa0da3-3ee1-4955-bc33-a835a337713b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:59:23.744731 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa0da3-3ee1-4955-bc33-a835a337713b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:23.746529 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa0da3-3ee1-4955-bc33-a835a337713b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:23.750095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e78da2-78ce-4511-9e3c-13d9f412a5b5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:23.750536 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e78da2-78ce-4511-9e3c-13d9f412a5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:23.750942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e78da2-78ce-4511-9e3c-13d9f412a5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:23.775689 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 853/1700] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:59:23.782146 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa0da3-3ee1-4955-bc33-a835a337713b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:23.787080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-84e78da2-78ce-4511-9e3c-13d9f412a5b5 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:23.791543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-84e78da2-78ce-4511-9e3c-13d9f412a5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:23.809661 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa0da3-3ee1-4955-bc33-a835a337713b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:23.942427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:24.002892 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d4472-59c2-4023-a530-231a813ae217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:24.003856 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d4472-59c2-4023-a530-231a813ae217 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:24.005708 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d4472-59c2-4023-a530-231a813ae217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:24.010141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d4472-59c2-4023-a530-231a813ae217 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:24.011180 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d4472-59c2-4023-a530-231a813ae217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:24.013686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d4472-59c2-4023-a530-231a813ae217 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:24.039619 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 848/1701] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:24.080088 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e78da2-78ce-4511-9e3c-13d9f412a5b5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:24.081711 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-84e78da2-78ce-4511-9e3c-13d9f412a5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:24.085057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 854/1702] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:23 2022] POST /identity/v3/projects => generated 387 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:24.088613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d4472-59c2-4023-a530-231a813ae217 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:24.247256 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e78da2-78ce-4511-9e3c-13d9f412a5b5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:24.307249 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d4472-59c2-4023-a530-231a813ae217 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:24.324114 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a32dc2-079a-4708-bbc2-94d1ec52b870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:24.324814 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a32dc2-079a-4708-bbc2-94d1ec52b870 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:24.325096 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a32dc2-079a-4708-bbc2-94d1ec52b870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:24.325510 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a32dc2-079a-4708-bbc2-94d1ec52b870 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:24.326195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a32dc2-079a-4708-bbc2-94d1ec52b870 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:24.328218 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a32dc2-079a-4708-bbc2-94d1ec52b870 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:24.373253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 855/1703] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:24 2022] POST /identity/v3/users => generated 294 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:59:24.379036 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a32dc2-079a-4708-bbc2-94d1ec52b870 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:24.391121 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a32dc2-079a-4708-bbc2-94d1ec52b870 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:24.396700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa0da3-fb35-49f1-9d6c-14b539ac5091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:24.397662 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa0da3-fb35-49f1-9d6c-14b539ac5091 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:24.398450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e39e16-883b-4cf9-9d31-f4d88c1b85e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:24.399307 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e39e16-883b-4cf9-9d31-f4d88c1b85e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:24.399598 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e39e16-883b-4cf9-9d31-f4d88c1b85e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:24.399876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e39e16-883b-4cf9-9d31-f4d88c1b85e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:24.400786 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e39e16-883b-4cf9-9d31-f4d88c1b85e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:24.401903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa0da3-fb35-49f1-9d6c-14b539ac5091 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:24.402195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa0da3-fb35-49f1-9d6c-14b539ac5091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:24.402924 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e39e16-883b-4cf9-9d31-f4d88c1b85e2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:24.417960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 856/1704] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:24.419057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e39e16-883b-4cf9-9d31-f4d88c1b85e2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:24.429653 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa0da3-fb35-49f1-9d6c-14b539ac5091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:24.431473 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa0da3-fb35-49f1-9d6c-14b539ac5091 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:24.434310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e39e16-883b-4cf9-9d31-f4d88c1b85e2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:24.459730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 849/1705] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:24.464826 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eefed0-b756-4402-800d-528fa1c0d0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:24.466664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eefed0-b756-4402-800d-528fa1c0d0ba admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:24.467383 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eefed0-b756-4402-800d-528fa1c0d0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:24.467943 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eefed0-b756-4402-800d-528fa1c0d0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:24.469448 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eefed0-b756-4402-800d-528fa1c0d0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:24.475399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eefed0-b756-4402-800d-528fa1c0d0ba admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:24.508788 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 857/1706] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:24.513896 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa0da3-fb35-49f1-9d6c-14b539ac5091 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:24.542676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa0da3-fb35-49f1-9d6c-14b539ac5091 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:24.570386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6944f04-a6ee-48a7-9918-6e04a642da28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:24.571298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6944f04-a6ee-48a7-9918-6e04a642da28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:24.572052 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6944f04-a6ee-48a7-9918-6e04a642da28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:24.574779 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6944f04-a6ee-48a7-9918-6e04a642da28 admin admin] PATH_INFO: `/v3/projects/38b4c11513084990aea4986063f9e181/users/cbb6c2361ac5492e9eaf0f9b864df97f/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:24.641086 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6944f04-a6ee-48a7-9918-6e04a642da28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b4c11513084990aea4986063f9e181, user_id=cbb6c2361ac5492e9eaf0f9b864df97f, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:24.644907 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6944f04-a6ee-48a7-9918-6e04a642da28 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:24.684738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 850/1707] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:24 2022] PUT /identity/v3/projects/38b4c11513084990aea4986063f9e181/users/cbb6c2361ac5492e9eaf0f9b864df97f/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:24.696748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c4dd1-8896-4707-945b-9197cf020b12 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:24.698156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c4dd1-8896-4707-945b-9197cf020b12 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:24.699234 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c4dd1-8896-4707-945b-9197cf020b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:24.716319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbbea01-fd4a-40a3-ae5d-1d4a8684e7ae None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:24.717233 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbbea01-fd4a-40a3-ae5d-1d4a8684e7ae None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:24.717901 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbbea01-fd4a-40a3-ae5d-1d4a8684e7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:24.763643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-030c4dd1-8896-4707-945b-9197cf020b12 None None] MFA Rules not processed for user `cbb6c2361ac5492e9eaf0f9b864df97f`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:24.765559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-030c4dd1-8896-4707-945b-9197cf020b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:24.777348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-cfbbea01-fd4a-40a3-ae5d-1d4a8684e7ae None None] MFA Rules not processed for user `7f703f74c7ff43a291f54103e164ccdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:24.779682 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-cfbbea01-fd4a-40a3-ae5d-1d4a8684e7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:24.868900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:24.926153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 858/1708] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:24.926153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8ae4e-1ca1-4075-b37e-5df2031717e7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:24.926153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8ae4e-1ca1-4075-b37e-5df2031717e7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:24.926153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8ae4e-1ca1-4075-b37e-5df2031717e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:24.961301 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e3f8ae4e-1ca1-4075-b37e-5df2031717e7 None None] MFA Rules not processed for user `e8813b3f6f8c46519fb827cf521bb37b`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:24.963336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e3f8ae4e-1ca1-4075-b37e-5df2031717e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:24.993706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 851/1709] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:59:24.998554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbbea01-fd4a-40a3-ae5d-1d4a8684e7ae None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:25.021340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:25.104388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 859/1710] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:25.104388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc073d0e-0cca-4107-950b-5c39ff3bf690 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:25.104388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc073d0e-0cca-4107-950b-5c39ff3bf690 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:25.104388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc073d0e-0cca-4107-950b-5c39ff3bf690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:25.133798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbbea01-fd4a-40a3-ae5d-1d4a8684e7ae None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:25.142732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-bc073d0e-0cca-4107-950b-5c39ff3bf690 None None] MFA Rules not processed for user `7f703f74c7ff43a291f54103e164ccdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:25.144017 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bc073d0e-0cca-4107-950b-5c39ff3bf690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:25.145581 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08db821-f294-4c25-9a4d-8f9226c545e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:25.146316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08db821-f294-4c25-9a4d-8f9226c545e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:25.146717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08db821-f294-4c25-9a4d-8f9226c545e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:25.148285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08db821-f294-4c25-9a4d-8f9226c545e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:25.149165 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08db821-f294-4c25-9a4d-8f9226c545e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:25.155858 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08db821-f294-4c25-9a4d-8f9226c545e8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:25.167877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 860/1711] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:25.177926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812ddbd-a9a9-4808-908b-32c4f8590f86 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:25.178219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812ddbd-a9a9-4808-908b-32c4f8590f86 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:25.178502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812ddbd-a9a9-4808-908b-32c4f8590f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:25.234859 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 852/1712] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:24 2022] POST /identity/v3/projects => generated 391 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:25.242600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08db821-f294-4c25-9a4d-8f9226c545e8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:25.247457 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f08db821-f294-4c25-9a4d-8f9226c545e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:25.267633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1812ddbd-a9a9-4808-908b-32c4f8590f86 None None] MFA Rules not processed for user `e8813b3f6f8c46519fb827cf521bb37b`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:25.273760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1812ddbd-a9a9-4808-908b-32c4f8590f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:25.479136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 861/1713] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:25.480575 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812ddbd-a9a9-4808-908b-32c4f8590f86 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:25.552194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08db821-f294-4c25-9a4d-8f9226c545e8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:25.562385 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730046eb-9733-4efd-b92b-3231a47bb234 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:25.563121 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730046eb-9733-4efd-b92b-3231a47bb234 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:25.565274 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730046eb-9733-4efd-b92b-3231a47bb234 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:25.569794 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730046eb-9733-4efd-b92b-3231a47bb234 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:25.677443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812ddbd-a9a9-4808-908b-32c4f8590f86 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:25.698717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa973f3a-5636-421f-8a5c-eb36a25784e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:25.700960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa973f3a-5636-421f-8a5c-eb36a25784e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:25.701263 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa973f3a-5636-421f-8a5c-eb36a25784e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:25.701546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa973f3a-5636-421f-8a5c-eb36a25784e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:25.702245 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa973f3a-5636-421f-8a5c-eb36a25784e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:25.721221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa973f3a-5636-421f-8a5c-eb36a25784e9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:25.744636 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:25.767979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 862/1714] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:25 2022] POST /identity/v3/users => generated 302 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:25.768701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa973f3a-5636-421f-8a5c-eb36a25784e9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:25.773327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-fa973f3a-5636-421f-8a5c-eb36a25784e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:25.809617 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730046eb-9733-4efd-b92b-3231a47bb234 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:25.815760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730046eb-9733-4efd-b92b-3231a47bb234 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:25.825367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa973f3a-5636-421f-8a5c-eb36a25784e9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:25.842090 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef2db7-bdb6-4571-8061-3835fbbb2571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:25.843132 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef2db7-bdb6-4571-8061-3835fbbb2571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:25.849276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef2db7-bdb6-4571-8061-3835fbbb2571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:25.849676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef2db7-bdb6-4571-8061-3835fbbb2571 admin admin] PATH_INFO: `/v3/projects/be909f4e75e64d0a862dde797fe3a522` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:25.852833 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef2db7-bdb6-4571-8061-3835fbbb2571 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be909f4e75e64d0a862dde797fe3a522)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:25.855121 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef2db7-bdb6-4571-8061-3835fbbb2571 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:25.857145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 853/1715] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:25.858318 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730046eb-9733-4efd-b92b-3231a47bb234 service nova] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:25.863546 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-730046eb-9733-4efd-b92b-3231a47bb234 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:25.951535 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730046eb-9733-4efd-b92b-3231a47bb234 service nova] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:25.990004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816faa7-979b-4e79-a126-be3ebdd6bb14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:25.993758 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816faa7-979b-4e79-a126-be3ebdd6bb14 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:25.994066 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816faa7-979b-4e79-a126-be3ebdd6bb14 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:25.994356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816faa7-979b-4e79-a126-be3ebdd6bb14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:26.030219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816faa7-979b-4e79-a126-be3ebdd6bb14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:26.036296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-84ef2db7-bdb6-4571-8061-3835fbbb2571 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be909f4e75e64d0a862dde797fe3a522', 'request_id': 'req-84ef2db7-bdb6-4571-8061-3835fbbb2571'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:26.037757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816faa7-979b-4e79-a126-be3ebdd6bb14 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:26.052216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-84ef2db7-bdb6-4571-8061-3835fbbb2571 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be909f4e75e64d0a862dde797fe3a522', 'request_id': 'req-84ef2db7-bdb6-4571-8061-3835fbbb2571'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:26.066665 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-84ef2db7-bdb6-4571-8061-3835fbbb2571 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be909f4e75e64d0a862dde797fe3a522', 'request_id': 'req-84ef2db7-bdb6-4571-8061-3835fbbb2571'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:26.090746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 854/1716] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:26.091698 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816faa7-979b-4e79-a126-be3ebdd6bb14 service nova] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:26.106186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-84ef2db7-bdb6-4571-8061-3835fbbb2571 admin admin] The token cache is being invalidate because project be909f4e75e64d0a862dde797fe3a522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:59:26.106674 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-84ef2db7-bdb6-4571-8061-3835fbbb2571 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:26.107999 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-84ef2db7-bdb6-4571-8061-3835fbbb2571 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:26.115178 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 863/1717] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 18:59:25 2022] DELETE /identity/v3/projects/be909f4e75e64d0a862dde797fe3a522 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:26.117115 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef2db7-bdb6-4571-8061-3835fbbb2571 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:26.135834 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816faa7-979b-4e79-a126-be3ebdd6bb14 service nova] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:26.139751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-84ef2db7-bdb6-4571-8061-3835fbbb2571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:26.179284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:26.354174 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef2db7-bdb6-4571-8061-3835fbbb2571 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:26.364304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7f05a-4cb0-4d8b-bcd6-b5a1ca4cc895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:26.365302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7f05a-4cb0-4d8b-bcd6-b5a1ca4cc895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:26.365655 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7f05a-4cb0-4d8b-bcd6-b5a1ca4cc895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:26.366000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7f05a-4cb0-4d8b-bcd6-b5a1ca4cc895 admin admin] PATH_INFO: `/v3/users/4516ee81b2d64072b5be7dcd5242f9ac` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:26.367860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7f05a-4cb0-4d8b-bcd6-b5a1ca4cc895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4516ee81b2d64072b5be7dcd5242f9ac)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:26.370265 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7f05a-4cb0-4d8b-bcd6-b5a1ca4cc895 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:26.517117 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f2a58c-2570-4700-aad2-2d767a39c1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:26.518180 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2a58c-2570-4700-aad2-2d767a39c1e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:26.518980 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2a58c-2570-4700-aad2-2d767a39c1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:26.525108 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2a58c-2570-4700-aad2-2d767a39c1e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:26.529026 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f2a58c-2570-4700-aad2-2d767a39c1e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:26.532005 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f2a58c-2570-4700-aad2-2d767a39c1e4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:26.595189 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 855/1718] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 504 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:26.601798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11f4e2-d67b-4e9b-8204-95fe9c32869b None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:26.602093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11f4e2-d67b-4e9b-8204-95fe9c32869b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:26.602485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11f4e2-d67b-4e9b-8204-95fe9c32869b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:26.626931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1aa7f05a-4cb0-4d8b-bcd6-b5a1ca4cc895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4516ee81b2d64072b5be7dcd5242f9ac'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:26.630671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1aa7f05a-4cb0-4d8b-bcd6-b5a1ca4cc895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4516ee81b2d64072b5be7dcd5242f9ac'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:26.631409 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1aa7f05a-4cb0-4d8b-bcd6-b5a1ca4cc895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4516ee81b2d64072b5be7dcd5242f9ac'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:26.653037 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1aa7f05a-4cb0-4d8b-bcd6-b5a1ca4cc895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4516ee81b2d64072b5be7dcd5242f9ac'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:26.676453 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-dc11f4e2-d67b-4e9b-8204-95fe9c32869b None None] MFA Rules not processed for user `04327f3d7b844ddb88cf4a81145634c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:26.678065 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-dc11f4e2-d67b-4e9b-8204-95fe9c32869b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:26.678917 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1aa7f05a-4cb0-4d8b-bcd6-b5a1ca4cc895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4516ee81b2d64072b5be7dcd5242f9ac'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:26.735106 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:INSERT=4,keystone:SELECT=80,keystone:DELETE=16 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:26.775763 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 864/1719] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:26 2022] DELETE /identity/v3/users/4516ee81b2d64072b5be7dcd5242f9ac => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:26.775763 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7f05a-4cb0-4d8b-bcd6-b5a1ca4cc895 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:26.777219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1aa7f05a-4cb0-4d8b-bcd6-b5a1ca4cc895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:26.852812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 856/1720] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:26.955946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7f05a-4cb0-4d8b-bcd6-b5a1ca4cc895 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:26.972864 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63d7db-3d98-4a39-afa3-5bc68dce85a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:26.974369 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63d7db-3d98-4a39-afa3-5bc68dce85a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:26.974369 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63d7db-3d98-4a39-afa3-5bc68dce85a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:26.974369 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63d7db-3d98-4a39-afa3-5bc68dce85a5 admin admin] PATH_INFO: `/v3/projects/43797666d9aa415db40e4a5160b5c3b9/users/cff6213cb24346de9dd9dca6d47b7f94/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:27.017900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63d7db-3d98-4a39-afa3-5bc68dce85a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43797666d9aa415db40e4a5160b5c3b9, user_id=cff6213cb24346de9dd9dca6d47b7f94, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:27.020648 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63d7db-3d98-4a39-afa3-5bc68dce85a5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:27.053339 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 865/1721] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:26 2022] PUT /identity/v3/projects/43797666d9aa415db40e4a5160b5c3b9/users/cff6213cb24346de9dd9dca6d47b7f94/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:27.061371 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11f4e2-d67b-4e9b-8204-95fe9c32869b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:27.078259 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11f4e2-d67b-4e9b-8204-95fe9c32869b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:27.268125 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c32a7-d234-4d08-a49c-06f23b6fe87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:27.269126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c32a7-d234-4d08-a49c-06f23b6fe87a admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:27.269557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c32a7-d234-4d08-a49c-06f23b6fe87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:27.269978 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c32a7-d234-4d08-a49c-06f23b6fe87a admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:27.270944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c32a7-d234-4d08-a49c-06f23b6fe87a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:27.272728 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c32a7-d234-4d08-a49c-06f23b6fe87a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:27.292582 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 857/1722] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:27.297616 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63d7db-3d98-4a39-afa3-5bc68dce85a5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:27.327639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63d7db-3d98-4a39-afa3-5bc68dce85a5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:27.336416 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbc56e-43f8-4570-876e-60045e94aee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:27.337850 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbc56e-43f8-4570-876e-60045e94aee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:27.339184 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbc56e-43f8-4570-876e-60045e94aee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:27.339892 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbc56e-43f8-4570-876e-60045e94aee7 admin admin] PATH_INFO: `/v3/projects/43797666d9aa415db40e4a5160b5c3b9/users/cff6213cb24346de9dd9dca6d47b7f94/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:27.343211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbc56e-43f8-4570-876e-60045e94aee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43797666d9aa415db40e4a5160b5c3b9, user_id=cff6213cb24346de9dd9dca6d47b7f94, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:27.347133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbc56e-43f8-4570-876e-60045e94aee7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:27.369013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 866/1723] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:27 2022] PUT /identity/v3/projects/43797666d9aa415db40e4a5160b5c3b9/users/cff6213cb24346de9dd9dca6d47b7f94/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:59:27.382343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97771fe9-5c48-4113-bf77-3681a52598e4 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:27.383086 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97771fe9-5c48-4113-bf77-3681a52598e4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:27.383653 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97771fe9-5c48-4113-bf77-3681a52598e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:27.473327 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-97771fe9-5c48-4113-bf77-3681a52598e4 None None] MFA Rules not processed for user `cff6213cb24346de9dd9dca6d47b7f94`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:27.475011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-97771fe9-5c48-4113-bf77-3681a52598e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:27.568461 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57fed2-6408-4bf8-95b3-5221c4c7a04e None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:27.568859 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57fed2-6408-4bf8-95b3-5221c4c7a04e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:27.569496 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57fed2-6408-4bf8-95b3-5221c4c7a04e None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:27.571561 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 867/1724] 10.222.0.70 () {54 vars in 912 bytes} [Thu Jun 16 18:59:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:59:27.579706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:27.588440 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32b39d-a5ec-4ce2-bdea-ee54b1ac24af None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:27.589447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32b39d-a5ec-4ce2-bdea-ee54b1ac24af None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:27.589907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32b39d-a5ec-4ce2-bdea-ee54b1ac24af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:27.727964 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5f32b39d-a5ec-4ce2-bdea-ee54b1ac24af None None] MFA Rules not processed for user `02a83f6e7aa448d6984a78914c1748cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:27.729173 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5f32b39d-a5ec-4ce2-bdea-ee54b1ac24af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:27.762924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 858/1725] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:27.775279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97771fe9-5c48-4113-bf77-3681a52598e4 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:27.777202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-97771fe9-5c48-4113-bf77-3681a52598e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:27.950997 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97771fe9-5c48-4113-bf77-3681a52598e4 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:27.960265 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 868/1726] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Jun 16 18:59:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:27.961306 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86434742-cdad-482d-8b41-1fcaa93fec2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:27.962403 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86434742-cdad-482d-8b41-1fcaa93fec2a service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:27.962818 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86434742-cdad-482d-8b41-1fcaa93fec2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:27.963217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86434742-cdad-482d-8b41-1fcaa93fec2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:27.980136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecf407-21a5-491e-835c-3d02add5531b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:27.981045 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecf407-21a5-491e-835c-3d02add5531b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:27.984366 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecf407-21a5-491e-835c-3d02add5531b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:27.986589 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 869/1727] 10.222.0.70 () {54 vars in 912 bytes} [Thu Jun 16 18:59:27 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:59:27.993625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e0fab-8b1b-487d-8706-a243188c724f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:27.999032 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e0fab-8b1b-487d-8706-a243188c724f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:27.999360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e0fab-8b1b-487d-8706-a243188c724f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:27.999786 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86434742-cdad-482d-8b41-1fcaa93fec2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:28.001927 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86434742-cdad-482d-8b41-1fcaa93fec2a service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:28.034909 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 859/1728] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:28.035931 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86434742-cdad-482d-8b41-1fcaa93fec2a service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:28.041066 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-86434742-cdad-482d-8b41-1fcaa93fec2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:28.050419 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-017e0fab-8b1b-487d-8706-a243188c724f None None] MFA Rules not processed for user `02a83f6e7aa448d6984a78914c1748cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:28.053528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-017e0fab-8b1b-487d-8706-a243188c724f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:28.096385 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86434742-cdad-482d-8b41-1fcaa93fec2a service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:28.096653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 870/1729] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Jun 16 18:59:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:28.103401 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751bcdee-95e4-4cd9-a420-b1141d0875e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:28.104332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bcdee-95e4-4cd9-a420-b1141d0875e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:28.104719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bcdee-95e4-4cd9-a420-b1141d0875e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:28.105096 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bcdee-95e4-4cd9-a420-b1141d0875e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:28.116661 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e0fab-8b1b-487d-8706-a243188c724f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:28.118141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-017e0fab-8b1b-487d-8706-a243188c724f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:28.132622 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751bcdee-95e4-4cd9-a420-b1141d0875e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:28.134341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751bcdee-95e4-4cd9-a420-b1141d0875e3 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:28.159483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 860/1730] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:28.193895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751bcdee-95e4-4cd9-a420-b1141d0875e3 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:28.196720 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-751bcdee-95e4-4cd9-a420-b1141d0875e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:28.240445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751bcdee-95e4-4cd9-a420-b1141d0875e3 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:28.258150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b5a53-e8e8-4265-8c0a-dcff9dd964b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:28.259312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b5a53-e8e8-4265-8c0a-dcff9dd964b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:28.259694 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b5a53-e8e8-4265-8c0a-dcff9dd964b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:28.260091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b5a53-e8e8-4265-8c0a-dcff9dd964b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:28.262887 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d97b5a53-e8e8-4265-8c0a-dcff9dd964b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:28.279107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:28.341311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e0fab-8b1b-487d-8706-a243188c724f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:28.369173 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6980a3-87de-4f7d-b3f7-c8654255aa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:28.370081 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6980a3-87de-4f7d-b3f7-c8654255aa75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:28.370364 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6980a3-87de-4f7d-b3f7-c8654255aa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:28.370666 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6980a3-87de-4f7d-b3f7-c8654255aa75 admin admin] PATH_INFO: `/v3/projects/a67a4534ebbd4d859ea4a0cd62761cc7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:28.372063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6980a3-87de-4f7d-b3f7-c8654255aa75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a67a4534ebbd4d859ea4a0cd62761cc7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:28.374069 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6980a3-87de-4f7d-b3f7-c8654255aa75 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:28.417421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b5a53-e8e8-4265-8c0a-dcff9dd964b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:28.419045 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b5a53-e8e8-4265-8c0a-dcff9dd964b9 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:28.443507 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 861/1731] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:28.444684 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b5a53-e8e8-4265-8c0a-dcff9dd964b9 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:28.451735 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d97b5a53-e8e8-4265-8c0a-dcff9dd964b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:28.477200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b5a53-e8e8-4265-8c0a-dcff9dd964b9 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:28.485072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e7b28-f6fb-4c9d-a442-65ca2c0af62d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:28.486114 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e7b28-f6fb-4c9d-a442-65ca2c0af62d service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:28.486605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e7b28-f6fb-4c9d-a442-65ca2c0af62d service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:28.487178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e7b28-f6fb-4c9d-a442-65ca2c0af62d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:28.530775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58e7b28-f6fb-4c9d-a442-65ca2c0af62d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:28.533830 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58e7b28-f6fb-4c9d-a442-65ca2c0af62d service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:28.567363 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 862/1732] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:28.588806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1c6980a3-87de-4f7d-b3f7-c8654255aa75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a67a4534ebbd4d859ea4a0cd62761cc7', 'request_id': 'req-1c6980a3-87de-4f7d-b3f7-c8654255aa75'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:28.604852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1c6980a3-87de-4f7d-b3f7-c8654255aa75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a67a4534ebbd4d859ea4a0cd62761cc7', 'request_id': 'req-1c6980a3-87de-4f7d-b3f7-c8654255aa75'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:28.619114 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1c6980a3-87de-4f7d-b3f7-c8654255aa75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a67a4534ebbd4d859ea4a0cd62761cc7', 'request_id': 'req-1c6980a3-87de-4f7d-b3f7-c8654255aa75'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:28.652762 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1c6980a3-87de-4f7d-b3f7-c8654255aa75 admin admin] The token cache is being invalidate because project a67a4534ebbd4d859ea4a0cd62761cc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:59:28.653188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1c6980a3-87de-4f7d-b3f7-c8654255aa75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:28.654724 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1c6980a3-87de-4f7d-b3f7-c8654255aa75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:28.656774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 871/1733] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 18:59:28 2022] DELETE /identity/v3/projects/a67a4534ebbd4d859ea4a0cd62761cc7 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:28.663057 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba3f21-dae1-4529-8e58-86114a002d29 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:28.663555 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba3f21-dae1-4529-8e58-86114a002d29 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:28.663654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba3f21-dae1-4529-8e58-86114a002d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:28.696053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-86ba3f21-dae1-4529-8e58-86114a002d29 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:28.697298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-86ba3f21-dae1-4529-8e58-86114a002d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:28.799113 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:28.874350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 863/1734] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:28.884489 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3caf36-769f-4196-a098-9d278cdfdd76 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:28.884934 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3caf36-769f-4196-a098-9d278cdfdd76 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:28.885332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3caf36-769f-4196-a098-9d278cdfdd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:28.936481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-bc3caf36-769f-4196-a098-9d278cdfdd76 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:28.939325 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bc3caf36-769f-4196-a098-9d278cdfdd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:28.983722 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba3f21-dae1-4529-8e58-86114a002d29 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:28.984346 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 872/1735] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:29.000386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba3f21-dae1-4529-8e58-86114a002d29 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:29.036448 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8488671-e200-4a6e-a79f-61b5fe444678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:29.037147 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8488671-e200-4a6e-a79f-61b5fe444678 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:29.037433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8488671-e200-4a6e-a79f-61b5fe444678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:29.037709 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8488671-e200-4a6e-a79f-61b5fe444678 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:29.038448 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8488671-e200-4a6e-a79f-61b5fe444678 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:59:29.038783 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8488671-e200-4a6e-a79f-61b5fe444678 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:29.051705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8488671-e200-4a6e-a79f-61b5fe444678 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:29.100611 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 864/1736] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:59:29.101814 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3caf36-769f-4196-a098-9d278cdfdd76 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:29.124593 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3caf36-769f-4196-a098-9d278cdfdd76 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:29.150173 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbc61b-31a9-4f4f-9327-a1fb60951de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:29.157710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbc61b-31a9-4f4f-9327-a1fb60951de6 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:29.159013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbc61b-31a9-4f4f-9327-a1fb60951de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:29.160071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbc61b-31a9-4f4f-9327-a1fb60951de6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:29.167003 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbc61b-31a9-4f4f-9327-a1fb60951de6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:29.172214 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbc61b-31a9-4f4f-9327-a1fb60951de6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:29.282614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 873/1737] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:29 2022] POST /identity/v3/projects => generated 367 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:29.283166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8488671-e200-4a6e-a79f-61b5fe444678 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:29.301807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8488671-e200-4a6e-a79f-61b5fe444678 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:29.495476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66085dbe-e0a0-4df4-abf6-6968052ccad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:29.496682 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66085dbe-e0a0-4df4-abf6-6968052ccad1 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:29.497273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66085dbe-e0a0-4df4-abf6-6968052ccad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:29.497878 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66085dbe-e0a0-4df4-abf6-6968052ccad1 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:29.499043 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66085dbe-e0a0-4df4-abf6-6968052ccad1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:29.504287 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66085dbe-e0a0-4df4-abf6-6968052ccad1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:29.542861 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46358db-32da-48d7-b623-baf9f66579c1 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:29.543936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46358db-32da-48d7-b623-baf9f66579c1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:29.544501 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46358db-32da-48d7-b623-baf9f66579c1 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:29.549577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 874/1738] 10.222.0.70 () {54 vars in 960 bytes} [Thu Jun 16 18:59:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:59:29.553165 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 865/1739] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:29 2022] POST /identity/v3/users => generated 284 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:29.556549 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66085dbe-e0a0-4df4-abf6-6968052ccad1 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:29.556959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28107c-4580-4b6f-8555-e54d6d581688 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:29.557261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28107c-4580-4b6f-8555-e54d6d581688 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:29.557906 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28107c-4580-4b6f-8555-e54d6d581688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:29.584357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66085dbe-e0a0-4df4-abf6-6968052ccad1 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:29.601778 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830c6fd-83ab-4f8c-81df-c6fb6de20fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:29.602985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8b28107c-4580-4b6f-8555-e54d6d581688 None None] MFA Rules not processed for user `880d0299c38c4eb897c669a512e599be`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:29.604229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830c6fd-83ab-4f8c-81df-c6fb6de20fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:29.604584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830c6fd-83ab-4f8c-81df-c6fb6de20fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:29.605607 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8b28107c-4580-4b6f-8555-e54d6d581688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:29.606074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830c6fd-83ab-4f8c-81df-c6fb6de20fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:29.616488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830c6fd-83ab-4f8c-81df-c6fb6de20fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:29.621058 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830c6fd-83ab-4f8c-81df-c6fb6de20fa7 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:29.653248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 866/1740] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:29.656714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830c6fd-83ab-4f8c-81df-c6fb6de20fa7 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:29.663779 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3830c6fd-83ab-4f8c-81df-c6fb6de20fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:29.786843 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 875/1741] 10.222.0.70 () {58 vars in 1057 bytes} [Thu Jun 16 18:59:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:29.799396 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28107c-4580-4b6f-8555-e54d6d581688 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:29.857979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28107c-4580-4b6f-8555-e54d6d581688 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:29.887156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f7d12-b916-40a0-845b-0aa9cbdf907d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:29.887857 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f7d12-b916-40a0-845b-0aa9cbdf907d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:29.888141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f7d12-b916-40a0-845b-0aa9cbdf907d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:29.910541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f7d12-b916-40a0-845b-0aa9cbdf907d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:29.911614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f7d12-b916-40a0-845b-0aa9cbdf907d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:29.917537 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f7d12-b916-40a0-845b-0aa9cbdf907d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:29.930179 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830c6fd-83ab-4f8c-81df-c6fb6de20fa7 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:29.938140 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506340a-8c20-40c9-80c6-f6d96a7ee770 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:29.939717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506340a-8c20-40c9-80c6-f6d96a7ee770 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:29.940214 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506340a-8c20-40c9-80c6-f6d96a7ee770 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:29.946403 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506340a-8c20-40c9-80c6-f6d96a7ee770 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:29.950771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7506340a-8c20-40c9-80c6-f6d96a7ee770 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:29.961750 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 876/1742] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:29.963562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f7d12-b916-40a0-845b-0aa9cbdf907d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:29.994126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f7d12-b916-40a0-845b-0aa9cbdf907d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:30.051482 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891bbfb4-341b-4998-8e09-e07dfe44d1e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '880d0299c38c4eb897c669a512e599be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:30.052383 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891bbfb4-341b-4998-8e09-e07dfe44d1e6 service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:30.054024 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891bbfb4-341b-4998-8e09-e07dfe44d1e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:30.054844 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891bbfb4-341b-4998-8e09-e07dfe44d1e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:30.068183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-891bbfb4-341b-4998-8e09-e07dfe44d1e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:30.076497 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:30.106337 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506340a-8c20-40c9-80c6-f6d96a7ee770 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:30.108114 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506340a-8c20-40c9-80c6-f6d96a7ee770 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:30.124375 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 867/1743] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:30.126442 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8f71f-648e-4da4-8af1-a1add260a024 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:30.127032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8f71f-648e-4da4-8af1-a1add260a024 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:30.127557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8f71f-648e-4da4-8af1-a1add260a024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:30.129062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891bbfb4-341b-4998-8e09-e07dfe44d1e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:30.130937 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891bbfb4-341b-4998-8e09-e07dfe44d1e6 service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:30.177255 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 877/1744] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:30.178432 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891bbfb4-341b-4998-8e09-e07dfe44d1e6 service placement] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:30.180371 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-891bbfb4-341b-4998-8e09-e07dfe44d1e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:30.206689 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-01a8f71f-648e-4da4-8af1-a1add260a024 None None] MFA Rules not processed for user `8742a73e9b4f46deb0b5c9e1756ffe2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:30.207938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-01a8f71f-648e-4da4-8af1-a1add260a024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:30.210759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891bbfb4-341b-4998-8e09-e07dfe44d1e6 service placement] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:30.230035 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc327a-fd7e-405e-ab22-ba04f880b255 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '880d0299c38c4eb897c669a512e599be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:30.230716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc327a-fd7e-405e-ab22-ba04f880b255 service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:30.231089 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc327a-fd7e-405e-ab22-ba04f880b255 service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:30.232986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc327a-fd7e-405e-ab22-ba04f880b255 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:30.265420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc327a-fd7e-405e-ab22-ba04f880b255 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:30.279906 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc327a-fd7e-405e-ab22-ba04f880b255 service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:30.336717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 878/1745] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:30.336717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc327a-fd7e-405e-ab22-ba04f880b255 service placement] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:30.379930 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc327a-fd7e-405e-ab22-ba04f880b255 service placement] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:30.405991 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d26573-4d09-4bff-a732-15637167cec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:30.407002 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d26573-4d09-4bff-a732-15637167cec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:30.408296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d26573-4d09-4bff-a732-15637167cec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:30.408659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d26573-4d09-4bff-a732-15637167cec0 admin admin] PATH_INFO: `/v3/projects/3103f2f9283348e5b30a8389146c4e89/users/e18ab4fdf21b43c1b305083110d64718/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:30.455572 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:30.464763 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 868/1746] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:30.467985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d26573-4d09-4bff-a732-15637167cec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3103f2f9283348e5b30a8389146c4e89, user_id=e18ab4fdf21b43c1b305083110d64718, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:30.472672 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358be100-b46f-45dd-a75a-6abecbf47568 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:30.474071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358be100-b46f-45dd-a75a-6abecbf47568 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:30.474464 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358be100-b46f-45dd-a75a-6abecbf47568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:30.476241 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d26573-4d09-4bff-a732-15637167cec0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:30.534683 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 879/1747] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:30 2022] PUT /identity/v3/projects/3103f2f9283348e5b30a8389146c4e89/users/e18ab4fdf21b43c1b305083110d64718/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:30.546376 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-358be100-b46f-45dd-a75a-6abecbf47568 None None] MFA Rules not processed for user `7f703f74c7ff43a291f54103e164ccdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:30.548409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-358be100-b46f-45dd-a75a-6abecbf47568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:30.559152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa78224-ce7b-42b9-a687-f769f35c1c1a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:30.563130 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa78224-ce7b-42b9-a687-f769f35c1c1a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:30.563993 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa78224-ce7b-42b9-a687-f769f35c1c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:30.767874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7aa78224-ce7b-42b9-a687-f769f35c1c1a None None] MFA Rules not processed for user `e18ab4fdf21b43c1b305083110d64718`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:30.769221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7aa78224-ce7b-42b9-a687-f769f35c1c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:30.861019 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 869/1748] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:30.869057 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358be100-b46f-45dd-a75a-6abecbf47568 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:31.111779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 880/1749] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:30 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 560 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:31.114751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa78224-ce7b-42b9-a687-f769f35c1c1a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:31.253148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358be100-b46f-45dd-a75a-6abecbf47568 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:31.278963 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba2d13-dc9a-4a1d-93e0-e014f6e11e4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:31.280642 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba2d13-dc9a-4a1d-93e0-e014f6e11e4c service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:31.280986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba2d13-dc9a-4a1d-93e0-e014f6e11e4c service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:31.281318 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba2d13-dc9a-4a1d-93e0-e014f6e11e4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:31.346602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:31.365024 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa78224-ce7b-42b9-a687-f769f35c1c1a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:31.378906 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370950a5-ae66-4eb0-8c2a-9a060fedd9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:31.390451 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370950a5-ae66-4eb0-8c2a-9a060fedd9db admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:31.392215 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370950a5-ae66-4eb0-8c2a-9a060fedd9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:31.395501 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370950a5-ae66-4eb0-8c2a-9a060fedd9db admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:31.396481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370950a5-ae66-4eb0-8c2a-9a060fedd9db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:31.399381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370950a5-ae66-4eb0-8c2a-9a060fedd9db admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:31.496675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba2d13-dc9a-4a1d-93e0-e014f6e11e4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:31.501645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba2d13-dc9a-4a1d-93e0-e014f6e11e4c service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:31.539165 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 870/1750] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:31.555602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 881/1751] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:31 2022] POST /identity/v3/projects => generated 369 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:31.556774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba2d13-dc9a-4a1d-93e0-e014f6e11e4c service nova] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:31.589075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba2d13-dc9a-4a1d-93e0-e014f6e11e4c service nova] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:31.846494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952803ea-7b07-433e-80bf-bdeb53cdafde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:31.851072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952803ea-7b07-433e-80bf-bdeb53cdafde admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:31.851758 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952803ea-7b07-433e-80bf-bdeb53cdafde admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:31.852100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952803ea-7b07-433e-80bf-bdeb53cdafde admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:31.856593 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952803ea-7b07-433e-80bf-bdeb53cdafde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:31.862732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952803ea-7b07-433e-80bf-bdeb53cdafde admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:31.936332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 871/1752] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:31 2022] POST /identity/v3/users => generated 291 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:31.943009 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370950a5-ae66-4eb0-8c2a-9a060fedd9db admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:31.981288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370950a5-ae66-4eb0-8c2a-9a060fedd9db admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:32.005397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8e21a-9208-4902-bed5-19be047bae3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:32.006095 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8e21a-9208-4902-bed5-19be047bae3e admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:32.006376 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8e21a-9208-4902-bed5-19be047bae3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:32.006694 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8e21a-9208-4902-bed5-19be047bae3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:32.007414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8e21a-9208-4902-bed5-19be047bae3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:32.014026 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8e21a-9208-4902-bed5-19be047bae3e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:32.043172 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 882/1753] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:32.044472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952803ea-7b07-433e-80bf-bdeb53cdafde admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:32.066910 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952803ea-7b07-433e-80bf-bdeb53cdafde admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:32.074875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69d39c-6ee5-4673-8663-12c1584f4f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:32.075896 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69d39c-6ee5-4673-8663-12c1584f4f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:32.076189 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69d39c-6ee5-4673-8663-12c1584f4f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:32.076468 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69d39c-6ee5-4673-8663-12c1584f4f1c admin admin] PATH_INFO: `/v3/projects/f8cd6af276ec42189f34b4b7f57e3b7a/users/e4d4a54d3dc64e93ba50b6aa784da99c/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:32.145984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69d39c-6ee5-4673-8663-12c1584f4f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8cd6af276ec42189f34b4b7f57e3b7a, user_id=e4d4a54d3dc64e93ba50b6aa784da99c, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:32.148748 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69d39c-6ee5-4673-8663-12c1584f4f1c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:32.186731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 872/1754] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:32 2022] PUT /identity/v3/projects/f8cd6af276ec42189f34b4b7f57e3b7a/users/e4d4a54d3dc64e93ba50b6aa784da99c/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:59:32.191129 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8e21a-9208-4902-bed5-19be047bae3e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:32.214703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8e21a-9208-4902-bed5-19be047bae3e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:32.458633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:32.499674 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc38ec-d1d2-487e-bd53-b455df59e193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:32.501213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc38ec-d1d2-487e-bd53-b455df59e193 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:32.514866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc38ec-d1d2-487e-bd53-b455df59e193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:32.515048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc38ec-d1d2-487e-bd53-b455df59e193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:32.515976 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc38ec-d1d2-487e-bd53-b455df59e193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:32.519902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc38ec-d1d2-487e-bd53-b455df59e193 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:32.545891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 883/1755] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:32.551504 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69d39c-6ee5-4673-8663-12c1584f4f1c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:32.571066 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69d39c-6ee5-4673-8663-12c1584f4f1c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:32.587179 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392e545-181d-4945-810b-21dde9d45caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:32.588570 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392e545-181d-4945-810b-21dde9d45caa admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:32.591237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392e545-181d-4945-810b-21dde9d45caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:32.593599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392e545-181d-4945-810b-21dde9d45caa admin admin] PATH_INFO: `/v3/projects/f8cd6af276ec42189f34b4b7f57e3b7a/users/e4d4a54d3dc64e93ba50b6aa784da99c/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:32.601583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392e545-181d-4945-810b-21dde9d45caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8cd6af276ec42189f34b4b7f57e3b7a, user_id=e4d4a54d3dc64e93ba50b6aa784da99c, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:32.611016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392e545-181d-4945-810b-21dde9d45caa admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:32.642182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 873/1756] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:32 2022] PUT /identity/v3/projects/f8cd6af276ec42189f34b4b7f57e3b7a/users/e4d4a54d3dc64e93ba50b6aa784da99c/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:32.671872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb024e-8f7e-4ad9-a2db-d4d6ab4c02b8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:32.673703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb024e-8f7e-4ad9-a2db-d4d6ab4c02b8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:32.674084 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb024e-8f7e-4ad9-a2db-d4d6ab4c02b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:32.747852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-97fb024e-8f7e-4ad9-a2db-d4d6ab4c02b8 None None] MFA Rules not processed for user `e4d4a54d3dc64e93ba50b6aa784da99c`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:32.749356 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-97fb024e-8f7e-4ad9-a2db-d4d6ab4c02b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:33.022237 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 884/1757] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:32 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:33.441093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392e545-181d-4945-810b-21dde9d45caa admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:33.444166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d392e545-181d-4945-810b-21dde9d45caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:33.614186 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392e545-181d-4945-810b-21dde9d45caa admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:33.644016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ead4c-d39b-4c9d-924a-c6bf86dc80f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:33.645003 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ead4c-d39b-4c9d-924a-c6bf86dc80f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:33.646606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ead4c-d39b-4c9d-924a-c6bf86dc80f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:33.647068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ead4c-d39b-4c9d-924a-c6bf86dc80f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:33.682533 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ead4c-d39b-4c9d-924a-c6bf86dc80f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:33.688433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ead4c-d39b-4c9d-924a-c6bf86dc80f1 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:33.728472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 874/1758] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:33 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:35.286771 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccd9e3-df93-4026-9c28-392b16d52833 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:35.287739 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccd9e3-df93-4026-9c28-392b16d52833 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:35.288886 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccd9e3-df93-4026-9c28-392b16d52833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:35.327575 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-20ccd9e3-df93-4026-9c28-392b16d52833 None None] MFA Rules not processed for user `02c826fc0baa480393df4c884c9e35dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:35.331195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-20ccd9e3-df93-4026-9c28-392b16d52833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:35.481518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 885/1759] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Jun 16 18:59:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:59:35.823913 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ead4c-d39b-4c9d-924a-c6bf86dc80f1 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:35.826058 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9f5ead4c-d39b-4c9d-924a-c6bf86dc80f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:35.850271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:35.981033 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ccd9e3-df93-4026-9c28-392b16d52833 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:35.982827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-20ccd9e3-df93-4026-9c28-392b16d52833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:36.047817 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ead4c-d39b-4c9d-924a-c6bf86dc80f1 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:36.073720 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8b02c-1586-430c-9870-36719849f176 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b22fe0ab0354385924923d0d443acae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:36.074855 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8b02c-1586-430c-9870-36719849f176 service swift] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:36.076617 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8b02c-1586-430c-9870-36719849f176 service swift] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:36.077154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8b02c-1586-430c-9870-36719849f176 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:36.117976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8b02c-1586-430c-9870-36719849f176 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:36.119850 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8b02c-1586-430c-9870-36719849f176 service swift] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:36.156202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 875/1760] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Jun 16 18:59:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 18:59:36.316214 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ccd9e3-df93-4026-9c28-392b16d52833 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:36.338443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a44340-1456-4c84-9dd3-2e8ffc9ab80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:36.340592 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a44340-1456-4c84-9dd3-2e8ffc9ab80e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:36.348945 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a44340-1456-4c84-9dd3-2e8ffc9ab80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:36.349531 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a44340-1456-4c84-9dd3-2e8ffc9ab80e admin admin] PATH_INFO: `/v3/users/cbb6c2361ac5492e9eaf0f9b864df97f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:36.361065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a44340-1456-4c84-9dd3-2e8ffc9ab80e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbb6c2361ac5492e9eaf0f9b864df97f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:36.372154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a44340-1456-4c84-9dd3-2e8ffc9ab80e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:36.494414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:36.683401 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-53a44340-1456-4c84-9dd3-2e8ffc9ab80e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbb6c2361ac5492e9eaf0f9b864df97f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:36.685352 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-53a44340-1456-4c84-9dd3-2e8ffc9ab80e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbb6c2361ac5492e9eaf0f9b864df97f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:36.693165 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-53a44340-1456-4c84-9dd3-2e8ffc9ab80e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbb6c2361ac5492e9eaf0f9b864df97f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:36.715236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-53a44340-1456-4c84-9dd3-2e8ffc9ab80e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbb6c2361ac5492e9eaf0f9b864df97f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:36.748923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-53a44340-1456-4c84-9dd3-2e8ffc9ab80e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbb6c2361ac5492e9eaf0f9b864df97f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:36.839663 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 886/1761] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:35 2022] DELETE /identity/v3/users/cbb6c2361ac5492e9eaf0f9b864df97f => generated 0 bytes in 856 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:36.841819 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409ef0a-5ee5-4bef-8cc7-df6870a35bb9 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:36.842847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409ef0a-5ee5-4bef-8cc7-df6870a35bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:36.843349 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409ef0a-5ee5-4bef-8cc7-df6870a35bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:36.947205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c409ef0a-5ee5-4bef-8cc7-df6870a35bb9 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:36.948911 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c409ef0a-5ee5-4bef-8cc7-df6870a35bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:37.320776 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 876/1762] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:37.504847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a44340-1456-4c84-9dd3-2e8ffc9ab80e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:37.506377 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-53a44340-1456-4c84-9dd3-2e8ffc9ab80e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:37.520250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409ef0a-5ee5-4bef-8cc7-df6870a35bb9 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:37.522108 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c409ef0a-5ee5-4bef-8cc7-df6870a35bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:37.571230 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a44340-1456-4c84-9dd3-2e8ffc9ab80e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:37.591481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbde3f0-a0e7-47e7-8857-94df0830adc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:37.597667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbde3f0-a0e7-47e7-8857-94df0830adc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:37.599582 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbde3f0-a0e7-47e7-8857-94df0830adc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:37.600660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbde3f0-a0e7-47e7-8857-94df0830adc7 admin admin] PATH_INFO: `/v3/users/0173edf0b4a44f6e9012194f2cd5f691` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:37.603296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbde3f0-a0e7-47e7-8857-94df0830adc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0173edf0b4a44f6e9012194f2cd5f691)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:37.613836 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbde3f0-a0e7-47e7-8857-94df0830adc7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:37.748866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1bbde3f0-a0e7-47e7-8857-94df0830adc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0173edf0b4a44f6e9012194f2cd5f691'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:37.750201 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1bbde3f0-a0e7-47e7-8857-94df0830adc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0173edf0b4a44f6e9012194f2cd5f691'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:37.755569 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1bbde3f0-a0e7-47e7-8857-94df0830adc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0173edf0b4a44f6e9012194f2cd5f691'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:37.771713 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1bbde3f0-a0e7-47e7-8857-94df0830adc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0173edf0b4a44f6e9012194f2cd5f691'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:37.786684 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409ef0a-5ee5-4bef-8cc7-df6870a35bb9 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:37.805860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1bbde3f0-a0e7-47e7-8857-94df0830adc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0173edf0b4a44f6e9012194f2cd5f691'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:37.817699 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1073657-4d93-459a-af36-5e5da51bb07c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:37.818734 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1073657-4d93-459a-af36-5e5da51bb07c service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:37.820096 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1073657-4d93-459a-af36-5e5da51bb07c service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:37.820563 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1073657-4d93-459a-af36-5e5da51bb07c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:37.827677 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a1073657-4d93-459a-af36-5e5da51bb07c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:37.831203 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 887/1763] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:37 2022] DELETE /identity/v3/users/0173edf0b4a44f6e9012194f2cd5f691 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:59:37.840093 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5c039-624c-4fd1-8720-e2e7fb660c22 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:37.840393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5c039-624c-4fd1-8720-e2e7fb660c22 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:37.840684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5c039-624c-4fd1-8720-e2e7fb660c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:37.870923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-86b5c039-624c-4fd1-8720-e2e7fb660c22 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:37.872165 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-86b5c039-624c-4fd1-8720-e2e7fb660c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:37.898536 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1073657-4d93-459a-af36-5e5da51bb07c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:37.901791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1073657-4d93-459a-af36-5e5da51bb07c service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:37.964798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 877/1764] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:37.984591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 888/1765] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:38.146146 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1073657-4d93-459a-af36-5e5da51bb07c service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:38.147650 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a1073657-4d93-459a-af36-5e5da51bb07c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:38.170066 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1073657-4d93-459a-af36-5e5da51bb07c service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:38.263508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:38.276316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d35a6-b6aa-4a14-96b0-9960b47482be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:38.277650 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d35a6-b6aa-4a14-96b0-9960b47482be service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:38.278296 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d35a6-b6aa-4a14-96b0-9960b47482be service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:38.278876 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d35a6-b6aa-4a14-96b0-9960b47482be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:38.297082 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d35a6-b6aa-4a14-96b0-9960b47482be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:38.300312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d35a6-b6aa-4a14-96b0-9960b47482be service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:38.332274 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 878/1766] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:38.703488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5c039-624c-4fd1-8720-e2e7fb660c22 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:38.706524 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-86b5c039-624c-4fd1-8720-e2e7fb660c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:38.732285 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5c039-624c-4fd1-8720-e2e7fb660c22 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:38.754262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14772c9-9c12-4615-87ab-384e99cc3f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:38.755444 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14772c9-9c12-4615-87ab-384e99cc3f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:38.755918 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14772c9-9c12-4615-87ab-384e99cc3f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:38.756340 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14772c9-9c12-4615-87ab-384e99cc3f97 admin admin] PATH_INFO: `/v3/users/e18ab4fdf21b43c1b305083110d64718` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:38.759176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14772c9-9c12-4615-87ab-384e99cc3f97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e18ab4fdf21b43c1b305083110d64718)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:38.764826 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14772c9-9c12-4615-87ab-384e99cc3f97 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:38.780659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:38.951357 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b14772c9-9c12-4615-87ab-384e99cc3f97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e18ab4fdf21b43c1b305083110d64718'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:38.955467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b14772c9-9c12-4615-87ab-384e99cc3f97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e18ab4fdf21b43c1b305083110d64718'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:38.957451 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b14772c9-9c12-4615-87ab-384e99cc3f97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e18ab4fdf21b43c1b305083110d64718'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:38.989628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b14772c9-9c12-4615-87ab-384e99cc3f97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e18ab4fdf21b43c1b305083110d64718'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:39.020343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b14772c9-9c12-4615-87ab-384e99cc3f97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e18ab4fdf21b43c1b305083110d64718'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:39.105615 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 889/1767] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:38 2022] DELETE /identity/v3/users/e18ab4fdf21b43c1b305083110d64718 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:39.108313 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664651d6-08c3-4c07-9e51-46aba67f4169 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:39.108847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664651d6-08c3-4c07-9e51-46aba67f4169 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:39.109072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664651d6-08c3-4c07-9e51-46aba67f4169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:39.147518 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-664651d6-08c3-4c07-9e51-46aba67f4169 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:39.160290 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-664651d6-08c3-4c07-9e51-46aba67f4169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:39.347938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 879/1768] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:39.518985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14772c9-9c12-4615-87ab-384e99cc3f97 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:39.521082 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b14772c9-9c12-4615-87ab-384e99cc3f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:39.668162 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14772c9-9c12-4615-87ab-384e99cc3f97 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:39.681268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32448c-38f9-4b87-a771-e98df0e840eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:39.683846 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32448c-38f9-4b87-a771-e98df0e840eb service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:39.685416 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32448c-38f9-4b87-a771-e98df0e840eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:39.687526 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32448c-38f9-4b87-a771-e98df0e840eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:39.729176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32448c-38f9-4b87-a771-e98df0e840eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:39.731244 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32448c-38f9-4b87-a771-e98df0e840eb service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:39.781136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 890/1769] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:40.970945 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fcd50-65e0-4da8-a1c0-441373f593a4 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:40.971552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fcd50-65e0-4da8-a1c0-441373f593a4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:40.971649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fcd50-65e0-4da8-a1c0-441373f593a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:41.021363 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f75fcd50-65e0-4da8-a1c0-441373f593a4 None None] MFA Rules not processed for user `02a83f6e7aa448d6984a78914c1748cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:41.024293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f75fcd50-65e0-4da8-a1c0-441373f593a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:41.050417 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 880/1770] 10.222.0.78 () {58 vars in 1015 bytes} [Thu Jun 16 18:59:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:41.090713 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32448c-38f9-4b87-a771-e98df0e840eb service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:41.092425 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bd32448c-38f9-4b87-a771-e98df0e840eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:41.130296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32448c-38f9-4b87-a771-e98df0e840eb service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:41.140482 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d03f6-efac-4c60-b83d-00f350f47e94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:41.141991 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d03f6-efac-4c60-b83d-00f350f47e94 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:41.142493 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d03f6-efac-4c60-b83d-00f350f47e94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:41.142863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d03f6-efac-4c60-b83d-00f350f47e94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:41.186480 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d03f6-efac-4c60-b83d-00f350f47e94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:41.197339 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d03f6-efac-4c60-b83d-00f350f47e94 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:41.240103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 891/1771] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:41.406011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a65a38-c01e-4b52-acf6-9e70ff3e0b3d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:41.407346 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a65a38-c01e-4b52-acf6-9e70ff3e0b3d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:41.408180 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a65a38-c01e-4b52-acf6-9e70ff3e0b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:41.442045 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-95a65a38-c01e-4b52-acf6-9e70ff3e0b3d None None] MFA Rules not processed for user `04327f3d7b844ddb88cf4a81145634c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:41.444183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-95a65a38-c01e-4b52-acf6-9e70ff3e0b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:41.588971 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 881/1772] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:41.616262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d03f6-efac-4c60-b83d-00f350f47e94 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:41.619078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5a0d03f6-efac-4c60-b83d-00f350f47e94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:41.654030 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d03f6-efac-4c60-b83d-00f350f47e94 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:41.671548 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:41.674410 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:41.674899 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:41.675346 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5 admin admin] PATH_INFO: `/v3/projects/df6eed19c14e457984f8bd65673c6134` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:41.676849 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df6eed19c14e457984f8bd65673c6134)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:41.679293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:41.862534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df6eed19c14e457984f8bd65673c6134', 'request_id': 'req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:41.880237 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df6eed19c14e457984f8bd65673c6134', 'request_id': 'req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:41.911130 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df6eed19c14e457984f8bd65673c6134', 'request_id': 'req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:41.926511 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:41.977365 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5 admin admin] The token cache is being invalidate because project df6eed19c14e457984f8bd65673c6134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:59:41.977822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:41.985151 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:41.991481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 892/1773] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 18:59:41 2022] DELETE /identity/v3/projects/df6eed19c14e457984f8bd65673c6134 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:41.993223 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca1af2-173a-4f5b-8b58-59d2afb3aefe None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:41.993732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca1af2-173a-4f5b-8b58-59d2afb3aefe None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:41.993824 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca1af2-173a-4f5b-8b58-59d2afb3aefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:42.051234 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-66ca1af2-173a-4f5b-8b58-59d2afb3aefe None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:42.052937 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-66ca1af2-173a-4f5b-8b58-59d2afb3aefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:42.150300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:42.156057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:42.303311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 882/1774] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:42.311117 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf579a-ba23-4829-875f-862d4990c6ea None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:42.311615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf579a-ba23-4829-875f-862d4990c6ea None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:42.312052 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf579a-ba23-4829-875f-862d4990c6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:42.372846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-03bf579a-ba23-4829-875f-862d4990c6ea None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:42.374012 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-03bf579a-ba23-4829-875f-862d4990c6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:42.381740 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b144a55-90a7-42ab-ac66-e4ea7fa1e7e5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:42.401851 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:42.402772 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:42.403057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:42.403334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517 admin admin] PATH_INFO: `/v3/projects/38b4c11513084990aea4986063f9e181` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:42.409922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 883/1775] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:42.410905 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b4c11513084990aea4986063f9e181)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:42.413944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf579a-ba23-4829-875f-862d4990c6ea None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:42.445103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:42.467355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf579a-ba23-4829-875f-862d4990c6ea None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:42.483404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192de87-4e95-4938-8a9c-32d92799e0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:42.484402 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192de87-4e95-4938-8a9c-32d92799e0bd admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:42.485574 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192de87-4e95-4938-8a9c-32d92799e0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:42.486109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192de87-4e95-4938-8a9c-32d92799e0bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:42.487218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4192de87-4e95-4938-8a9c-32d92799e0bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:59:42.487689 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4192de87-4e95-4938-8a9c-32d92799e0bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:42.489637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4192de87-4e95-4938-8a9c-32d92799e0bd admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:42.518207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 884/1776] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:59:42.523791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192de87-4e95-4938-8a9c-32d92799e0bd admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:42.548917 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:42.559285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192de87-4e95-4938-8a9c-32d92799e0bd admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:42.581750 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b4c11513084990aea4986063f9e181', 'request_id': 'req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:42.599429 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b4c11513084990aea4986063f9e181', 'request_id': 'req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:42.642331 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b4c11513084990aea4986063f9e181', 'request_id': 'req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:42.677239 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517 admin admin] The token cache is being invalidate because project 38b4c11513084990aea4986063f9e181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:59:42.677536 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:42.678381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:42.687515 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 893/1777] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 18:59:42 2022] DELETE /identity/v3/projects/38b4c11513084990aea4986063f9e181 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:42.688867 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:42.692905 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:42.790968 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2c428-1ee0-420e-ba43-fbd5ae6fdf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:42.792218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2c428-1ee0-420e-ba43-fbd5ae6fdf97 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:42.792506 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2c428-1ee0-420e-ba43-fbd5ae6fdf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:42.792785 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2c428-1ee0-420e-ba43-fbd5ae6fdf97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:42.793492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb2c428-1ee0-420e-ba43-fbd5ae6fdf97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:42.795467 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb2c428-1ee0-420e-ba43-fbd5ae6fdf97 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:42.888967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 885/1778] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:42 2022] POST /identity/v3/projects => generated 359 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:59:42.890517 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2c428-1ee0-420e-ba43-fbd5ae6fdf97 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:42.895250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8fb2c428-1ee0-420e-ba43-fbd5ae6fdf97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:42.897806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc394c-22ea-4e3f-98f4-133d1cf9f517 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:42.957369 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:43.012128 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef6ce4-379a-4a24-a73d-8f9b28717690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:43.012989 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef6ce4-379a-4a24-a73d-8f9b28717690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:43.013278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef6ce4-379a-4a24-a73d-8f9b28717690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:43.013917 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef6ce4-379a-4a24-a73d-8f9b28717690 admin admin] PATH_INFO: `/v3/users/cff6213cb24346de9dd9dca6d47b7f94` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:43.014933 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef6ce4-379a-4a24-a73d-8f9b28717690 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cff6213cb24346de9dd9dca6d47b7f94)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:43.022634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef6ce4-379a-4a24-a73d-8f9b28717690 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:43.140773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2c428-1ee0-420e-ba43-fbd5ae6fdf97 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:43.150748 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6470d79-eeab-4ff9-8fc0-a4238aea2cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:43.152647 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6470d79-eeab-4ff9-8fc0-a4238aea2cd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:43.153101 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6470d79-eeab-4ff9-8fc0-a4238aea2cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:43.153528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6470d79-eeab-4ff9-8fc0-a4238aea2cd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:43.156332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6470d79-eeab-4ff9-8fc0-a4238aea2cd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:43.162029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6470d79-eeab-4ff9-8fc0-a4238aea2cd8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:43.184331 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-37ef6ce4-379a-4a24-a73d-8f9b28717690 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cff6213cb24346de9dd9dca6d47b7f94'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:43.185278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-37ef6ce4-379a-4a24-a73d-8f9b28717690 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cff6213cb24346de9dd9dca6d47b7f94'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:43.186145 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-37ef6ce4-379a-4a24-a73d-8f9b28717690 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cff6213cb24346de9dd9dca6d47b7f94'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:43.214875 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-37ef6ce4-379a-4a24-a73d-8f9b28717690 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cff6213cb24346de9dd9dca6d47b7f94'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:43.227172 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 886/1779] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:42 2022] POST /identity/v3/users => generated 280 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:59:43.228986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6470d79-eeab-4ff9-8fc0-a4238aea2cd8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:43.234341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b6470d79-eeab-4ff9-8fc0-a4238aea2cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:43.269279 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-37ef6ce4-379a-4a24-a73d-8f9b28717690 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cff6213cb24346de9dd9dca6d47b7f94'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:43.274744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6470d79-eeab-4ff9-8fc0-a4238aea2cd8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:43.291443 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc4327-d3b4-4c19-8e76-d2b8d3c29b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:43.292500 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc4327-d3b4-4c19-8e76-d2b8d3c29b19 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:43.292954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc4327-d3b4-4c19-8e76-d2b8d3c29b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:43.293462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc4327-d3b4-4c19-8e76-d2b8d3c29b19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:43.294424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc4327-d3b4-4c19-8e76-d2b8d3c29b19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:43.299779 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc4327-d3b4-4c19-8e76-d2b8d3c29b19 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:43.331993 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 894/1780] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:42 2022] DELETE /identity/v3/users/cff6213cb24346de9dd9dca6d47b7f94 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:43.332951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 887/1781] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:43.339581 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef6ce4-379a-4a24-a73d-8f9b28717690 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:43.581738 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef6ce4-379a-4a24-a73d-8f9b28717690 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:43.598363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd365ab-cf07-4607-9111-5eb0669a343f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:43.599078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd365ab-cf07-4607-9111-5eb0669a343f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:43.599377 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd365ab-cf07-4607-9111-5eb0669a343f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:43.599692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd365ab-cf07-4607-9111-5eb0669a343f admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:43.600874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd365ab-cf07-4607-9111-5eb0669a343f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:43.605997 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd365ab-cf07-4607-9111-5eb0669a343f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:43.632168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc4327-d3b4-4c19-8e76-d2b8d3c29b19 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:43.634994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 895/1782] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:43.653181 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc4327-d3b4-4c19-8e76-d2b8d3c29b19 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:43.673710 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c5c50-64c1-436c-9fb1-4a4cfeb82b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:43.674720 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c5c50-64c1-436c-9fb1-4a4cfeb82b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:43.676653 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c5c50-64c1-436c-9fb1-4a4cfeb82b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:43.677553 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c5c50-64c1-436c-9fb1-4a4cfeb82b50 admin admin] PATH_INFO: `/v3/projects/8783e2da2acb4c6cb8d81356adf11816/users/ed0814cf31324178909d0075ff508db2/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:43.679236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038ee73-35b3-4e8b-8781-bf1026ee187a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:43.680427 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038ee73-35b3-4e8b-8781-bf1026ee187a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:43.681032 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038ee73-35b3-4e8b-8781-bf1026ee187a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:43.721390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c5c50-64c1-436c-9fb1-4a4cfeb82b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8783e2da2acb4c6cb8d81356adf11816, user_id=ed0814cf31324178909d0075ff508db2, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:43.730439 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0038ee73-35b3-4e8b-8781-bf1026ee187a None None] MFA Rules not processed for user `8742a73e9b4f46deb0b5c9e1756ffe2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:43.732929 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c5c50-64c1-436c-9fb1-4a4cfeb82b50 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:43.736984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0038ee73-35b3-4e8b-8781-bf1026ee187a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:43.783135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 888/1783] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:43 2022] PUT /identity/v3/projects/8783e2da2acb4c6cb8d81356adf11816/users/ed0814cf31324178909d0075ff508db2/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:59:43.822374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814106f-29a9-4b6f-b108-3c268d7d3c35 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:43.822374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814106f-29a9-4b6f-b108-3c268d7d3c35 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:43.822374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814106f-29a9-4b6f-b108-3c268d7d3c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:43.927223 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f814106f-29a9-4b6f-b108-3c268d7d3c35 None None] MFA Rules not processed for user `ed0814cf31324178909d0075ff508db2`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:43.932945 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f814106f-29a9-4b6f-b108-3c268d7d3c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:43.950381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:44.002745 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 896/1784] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:44.004517 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038ee73-35b3-4e8b-8781-bf1026ee187a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:44.009703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0038ee73-35b3-4e8b-8781-bf1026ee187a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:44.068821 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038ee73-35b3-4e8b-8781-bf1026ee187a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:44.188251 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:44.298628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:44.302497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:44.303194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:44.304241 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96 admin admin] PATH_INFO: `/v3/projects/3103f2f9283348e5b30a8389146c4e89` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:44.308313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3103f2f9283348e5b30a8389146c4e89)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:44.313572 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:44.315874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 889/1785] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:43 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 515 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:44.375458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814106f-29a9-4b6f-b108-3c268d7d3c35 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:44.377850 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f814106f-29a9-4b6f-b108-3c268d7d3c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:44.491629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3103f2f9283348e5b30a8389146c4e89', 'request_id': 'req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:44.516937 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3103f2f9283348e5b30a8389146c4e89', 'request_id': 'req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:44.531791 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3103f2f9283348e5b30a8389146c4e89', 'request_id': 'req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:44.566926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96 admin admin] The token cache is being invalidate because project 3103f2f9283348e5b30a8389146c4e89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:59:44.567203 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:44.570158 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:44.577700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 897/1786] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 18:59:44 2022] DELETE /identity/v3/projects/3103f2f9283348e5b30a8389146c4e89 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:44.579338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:44.582532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:44.623014 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814106f-29a9-4b6f-b108-3c268d7d3c35 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:44.797373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b82ce-25c3-4498-89aa-d41cbc3bea96 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:44.814810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a005ffd-64c7-4eb8-a8e9-c4eebd5092eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:44.826279 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005ffd-64c7-4eb8-a8e9-c4eebd5092eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:44.826819 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005ffd-64c7-4eb8-a8e9-c4eebd5092eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:44.831671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005ffd-64c7-4eb8-a8e9-c4eebd5092eb admin admin] PATH_INFO: `/v3/users/e4d4a54d3dc64e93ba50b6aa784da99c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:44.845394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a005ffd-64c7-4eb8-a8e9-c4eebd5092eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4d4a54d3dc64e93ba50b6aa784da99c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:44.860931 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc696b6d-35bd-43ba-b13d-81cd187279e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:44.861786 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc696b6d-35bd-43ba-b13d-81cd187279e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:44.862176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc696b6d-35bd-43ba-b13d-81cd187279e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:44.862562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc696b6d-35bd-43ba-b13d-81cd187279e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:44.864103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a005ffd-64c7-4eb8-a8e9-c4eebd5092eb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:44.877118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-bc696b6d-35bd-43ba-b13d-81cd187279e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:44.976048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:45.027828 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6a005ffd-64c7-4eb8-a8e9-c4eebd5092eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4d4a54d3dc64e93ba50b6aa784da99c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:45.029766 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6a005ffd-64c7-4eb8-a8e9-c4eebd5092eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4d4a54d3dc64e93ba50b6aa784da99c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:45.031335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6a005ffd-64c7-4eb8-a8e9-c4eebd5092eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4d4a54d3dc64e93ba50b6aa784da99c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:45.055262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6a005ffd-64c7-4eb8-a8e9-c4eebd5092eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4d4a54d3dc64e93ba50b6aa784da99c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:45.091378 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6a005ffd-64c7-4eb8-a8e9-c4eebd5092eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4d4a54d3dc64e93ba50b6aa784da99c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:45.138734 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 898/1787] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:44 2022] DELETE /identity/v3/users/e4d4a54d3dc64e93ba50b6aa784da99c => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:45.139890 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a005ffd-64c7-4eb8-a8e9-c4eebd5092eb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:45.145181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6a005ffd-64c7-4eb8-a8e9-c4eebd5092eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:45.216558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc696b6d-35bd-43ba-b13d-81cd187279e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:45.224125 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc696b6d-35bd-43ba-b13d-81cd187279e4 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:45.275073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 890/1788] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:44 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 900 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:45.290392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc696b6d-35bd-43ba-b13d-81cd187279e4 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:45.292892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-bc696b6d-35bd-43ba-b13d-81cd187279e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:45.311532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a005ffd-64c7-4eb8-a8e9-c4eebd5092eb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:45.319797 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f28d2-eb68-4d13-9df6-2b8e4678d81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:45.322143 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f28d2-eb68-4d13-9df6-2b8e4678d81a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:45.322968 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f28d2-eb68-4d13-9df6-2b8e4678d81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:45.323736 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f28d2-eb68-4d13-9df6-2b8e4678d81a admin admin] PATH_INFO: `/v3/users/c83a0e48174b44f5916004cd83487b34` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:45.325483 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f28d2-eb68-4d13-9df6-2b8e4678d81a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c83a0e48174b44f5916004cd83487b34)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:45.328162 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f28d2-eb68-4d13-9df6-2b8e4678d81a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:45.348262 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc696b6d-35bd-43ba-b13d-81cd187279e4 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:45.366357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950566f-2d10-4fd7-811f-a0f4abef8194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:45.367866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950566f-2d10-4fd7-811f-a0f4abef8194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:45.371102 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950566f-2d10-4fd7-811f-a0f4abef8194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:45.371493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950566f-2d10-4fd7-811f-a0f4abef8194 admin admin] PATH_INFO: `/v3/users/ed0814cf31324178909d0075ff508db2` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:45.376615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950566f-2d10-4fd7-811f-a0f4abef8194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed0814cf31324178909d0075ff508db2)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:45.379515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950566f-2d10-4fd7-811f-a0f4abef8194 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:45.418401 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d52f28d2-eb68-4d13-9df6-2b8e4678d81a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c83a0e48174b44f5916004cd83487b34'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:45.420404 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d52f28d2-eb68-4d13-9df6-2b8e4678d81a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c83a0e48174b44f5916004cd83487b34'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:45.421361 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d52f28d2-eb68-4d13-9df6-2b8e4678d81a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c83a0e48174b44f5916004cd83487b34'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:45.441727 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d52f28d2-eb68-4d13-9df6-2b8e4678d81a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c83a0e48174b44f5916004cd83487b34'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:45.465747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d52f28d2-eb68-4d13-9df6-2b8e4678d81a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c83a0e48174b44f5916004cd83487b34'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:45.506522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:45.528871 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d950566f-2d10-4fd7-811f-a0f4abef8194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed0814cf31324178909d0075ff508db2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:45.529274 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 899/1789] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:45 2022] DELETE /identity/v3/users/c83a0e48174b44f5916004cd83487b34 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:45.530291 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d950566f-2d10-4fd7-811f-a0f4abef8194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed0814cf31324178909d0075ff508db2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:45.533413 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cce9c-2607-42aa-99fb-7d509ad64d4b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:45.533710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cce9c-2607-42aa-99fb-7d509ad64d4b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:45.533994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cce9c-2607-42aa-99fb-7d509ad64d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:45.547830 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d950566f-2d10-4fd7-811f-a0f4abef8194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed0814cf31324178909d0075ff508db2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:45.568311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5a6cce9c-2607-42aa-99fb-7d509ad64d4b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:45.572052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5a6cce9c-2607-42aa-99fb-7d509ad64d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:45.580421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d950566f-2d10-4fd7-811f-a0f4abef8194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed0814cf31324178909d0075ff508db2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:45.581357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d950566f-2d10-4fd7-811f-a0f4abef8194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed0814cf31324178909d0075ff508db2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:45.611770 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 891/1790] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:45 2022] DELETE /identity/v3/users/ed0814cf31324178909d0075ff508db2 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:45.616526 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f4cb5-0351-43f1-bf64-27b03b153926 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:45.618160 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f4cb5-0351-43f1-bf64-27b03b153926 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:45.618672 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f4cb5-0351-43f1-bf64-27b03b153926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:45.630293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:45.678628 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-666f4cb5-0351-43f1-bf64-27b03b153926 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:45.684591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-666f4cb5-0351-43f1-bf64-27b03b153926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:45.762822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 900/1791] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:45.817626 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cce9c-2607-42aa-99fb-7d509ad64d4b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:45.823895 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5a6cce9c-2607-42aa-99fb-7d509ad64d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:45.904622 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 892/1792] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:45.941830 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f4cb5-0351-43f1-bf64-27b03b153926 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:45.943452 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-666f4cb5-0351-43f1-bf64-27b03b153926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:46.012152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cce9c-2607-42aa-99fb-7d509ad64d4b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:46.116814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f4cb5-0351-43f1-bf64-27b03b153926 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:46.127494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fff45-d5ec-447b-ab25-e336b20e30ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:46.128253 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fff45-d5ec-447b-ab25-e336b20e30ee service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:46.129858 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fff45-d5ec-447b-ab25-e336b20e30ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:46.130149 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fff45-d5ec-447b-ab25-e336b20e30ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:46.148465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7b7a1-a474-471f-a769-d46575a17c6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:46.149137 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7b7a1-a474-471f-a769-d46575a17c6c service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:46.153949 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7b7a1-a474-471f-a769-d46575a17c6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:46.158486 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7b7a1-a474-471f-a769-d46575a17c6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:46.167563 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3fff45-d5ec-447b-ab25-e336b20e30ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:46.178979 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3fff45-d5ec-447b-ab25-e336b20e30ee service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:46.223722 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7b7a1-a474-471f-a769-d46575a17c6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:46.231394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7b7a1-a474-471f-a769-d46575a17c6c service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:46.262476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 893/1793] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:59:46.297648 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 901/1794] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:46.399798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fff45-d5ec-447b-ab25-e336b20e30ee service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:46.404208 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9e3fff45-d5ec-447b-ab25-e336b20e30ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:46.472776 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fff45-d5ec-447b-ab25-e336b20e30ee service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:46.499634 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155c8c5-58d4-46f0-ab91-2e9a2b6e1093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:46.506317 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155c8c5-58d4-46f0-ab91-2e9a2b6e1093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:46.507003 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155c8c5-58d4-46f0-ab91-2e9a2b6e1093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:46.507561 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155c8c5-58d4-46f0-ab91-2e9a2b6e1093 admin admin] PATH_INFO: `/v3/projects/43797666d9aa415db40e4a5160b5c3b9` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:46.517555 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155c8c5-58d4-46f0-ab91-2e9a2b6e1093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43797666d9aa415db40e4a5160b5c3b9)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:46.531660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155c8c5-58d4-46f0-ab91-2e9a2b6e1093 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:46.811537 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a155c8c5-58d4-46f0-ab91-2e9a2b6e1093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43797666d9aa415db40e4a5160b5c3b9', 'request_id': 'req-a155c8c5-58d4-46f0-ab91-2e9a2b6e1093'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:46.817830 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:46.828732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a155c8c5-58d4-46f0-ab91-2e9a2b6e1093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43797666d9aa415db40e4a5160b5c3b9', 'request_id': 'req-a155c8c5-58d4-46f0-ab91-2e9a2b6e1093'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:46.858552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a155c8c5-58d4-46f0-ab91-2e9a2b6e1093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43797666d9aa415db40e4a5160b5c3b9', 'request_id': 'req-a155c8c5-58d4-46f0-ab91-2e9a2b6e1093'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:46.899724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a155c8c5-58d4-46f0-ab91-2e9a2b6e1093 admin admin] The token cache is being invalidate because project 43797666d9aa415db40e4a5160b5c3b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:59:46.900082 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a155c8c5-58d4-46f0-ab91-2e9a2b6e1093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:46.902344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a155c8c5-58d4-46f0-ab91-2e9a2b6e1093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:46.911751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9d893-eb3d-405b-b615-c329011847d9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:46.912352 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9d893-eb3d-405b-b615-c329011847d9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:46.912608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9d893-eb3d-405b-b615-c329011847d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:46.920891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 894/1795] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 18:59:46 2022] DELETE /identity/v3/projects/43797666d9aa415db40e4a5160b5c3b9 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:46.971348 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-80c9d893-eb3d-405b-b615-c329011847d9 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:46.978005 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-80c9d893-eb3d-405b-b615-c329011847d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:47.157787 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 902/1796] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:59:47.165971 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b6695-66cb-4a02-9ed6-442889e8091c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:47.166863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b6695-66cb-4a02-9ed6-442889e8091c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:47.167348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b6695-66cb-4a02-9ed6-442889e8091c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:47.201301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b61b6695-66cb-4a02-9ed6-442889e8091c None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:47.208056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b61b6695-66cb-4a02-9ed6-442889e8091c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:47.233660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 895/1797] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:47.235534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9d893-eb3d-405b-b615-c329011847d9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:47.253034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9d893-eb3d-405b-b615-c329011847d9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:47.269155 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42a19f-c841-45ae-ae5c-f30dec75698d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:47.270849 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42a19f-c841-45ae-ae5c-f30dec75698d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:47.270849 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42a19f-c841-45ae-ae5c-f30dec75698d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:47.270849 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42a19f-c841-45ae-ae5c-f30dec75698d admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:47.271673 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42a19f-c841-45ae-ae5c-f30dec75698d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:59:47.276597 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42a19f-c841-45ae-ae5c-f30dec75698d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:47.279207 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42a19f-c841-45ae-ae5c-f30dec75698d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:47.332322 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 903/1798] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:59:47.336640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b6695-66cb-4a02-9ed6-442889e8091c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:47.353598 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b6695-66cb-4a02-9ed6-442889e8091c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:47.369346 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9cf2f-1014-4ae6-b0a0-32cae2d2b3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:47.371865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9cf2f-1014-4ae6-b0a0-32cae2d2b3ec admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:47.381084 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9cf2f-1014-4ae6-b0a0-32cae2d2b3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:47.386037 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9cf2f-1014-4ae6-b0a0-32cae2d2b3ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:47.398902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9cf2f-1014-4ae6-b0a0-32cae2d2b3ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:47.422844 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9cf2f-1014-4ae6-b0a0-32cae2d2b3ec admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:47.553778 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 896/1799] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:47 2022] POST /identity/v3/projects => generated 399 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:47.555355 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42a19f-c841-45ae-ae5c-f30dec75698d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:47.596590 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9cf2f-1014-4ae6-b0a0-32cae2d2b3ec admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:47.599307 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42a19f-c841-45ae-ae5c-f30dec75698d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:47.607165 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8ed9cf2f-1014-4ae6-b0a0-32cae2d2b3ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:47.632689 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:47.825562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262ce29-5cf0-490c-8f01-4ca56a1bfbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:47.828002 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262ce29-5cf0-490c-8f01-4ca56a1bfbc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:47.829039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262ce29-5cf0-490c-8f01-4ca56a1bfbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:47.829918 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262ce29-5cf0-490c-8f01-4ca56a1bfbc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:47.832679 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262ce29-5cf0-490c-8f01-4ca56a1bfbc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:47.838458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262ce29-5cf0-490c-8f01-4ca56a1bfbc1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:47.935463 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 904/1800] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:47 2022] POST /identity/v3/users => generated 300 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:59:47.989527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c972b6-a742-4dd3-aaa0-773fdea8a075 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:47.989768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c972b6-a742-4dd3-aaa0-773fdea8a075 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:47.990029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c972b6-a742-4dd3-aaa0-773fdea8a075 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:47.993171 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 905/1801] 10.222.0.70 () {54 vars in 921 bytes} [Thu Jun 16 18:59:47 2022] GET /identity => generated 270 bytes in 57 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:59:47.998684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c972b6-a742-4dd3-aaa0-773fdea8a075 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:48.044678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c972b6-a742-4dd3-aaa0-773fdea8a075 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:48.059520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3b898-d5b9-4dcf-8900-f74cb3870ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:48.061392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3b898-d5b9-4dcf-8900-f74cb3870ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:48.062836 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3b898-d5b9-4dcf-8900-f74cb3870ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:48.064667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3b898-d5b9-4dcf-8900-f74cb3870ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:48.070558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c3b898-d5b9-4dcf-8900-f74cb3870ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:48.074326 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c3b898-d5b9-4dcf-8900-f74cb3870ae4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:48.158561 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 906/1802] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:48.161245 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f357fe4-6844-4bc1-8a46-0a5c949e1331 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:48.161890 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f357fe4-6844-4bc1-8a46-0a5c949e1331 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:48.162540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f357fe4-6844-4bc1-8a46-0a5c949e1331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:48.186668 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9cf2f-1014-4ae6-b0a0-32cae2d2b3ec admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:48.197645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62726a4-1792-4fa2-b05a-ca6f072175c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:48.198889 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62726a4-1792-4fa2-b05a-ca6f072175c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:48.199233 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62726a4-1792-4fa2-b05a-ca6f072175c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:48.199616 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62726a4-1792-4fa2-b05a-ca6f072175c7 admin admin] PATH_INFO: `/v3/projects/f8cd6af276ec42189f34b4b7f57e3b7a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:48.202898 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62726a4-1792-4fa2-b05a-ca6f072175c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8cd6af276ec42189f34b4b7f57e3b7a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:48.205974 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62726a4-1792-4fa2-b05a-ca6f072175c7 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:48.273839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9f357fe4-6844-4bc1-8a46-0a5c949e1331 None None] MFA Rules not processed for user `8342da838fe4462b9de2817ad3ac4ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:48.281502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9f357fe4-6844-4bc1-8a46-0a5c949e1331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:48.356179 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f62726a4-1792-4fa2-b05a-ca6f072175c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8cd6af276ec42189f34b4b7f57e3b7a', 'request_id': 'req-f62726a4-1792-4fa2-b05a-ca6f072175c7'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:48.376299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f62726a4-1792-4fa2-b05a-ca6f072175c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8cd6af276ec42189f34b4b7f57e3b7a', 'request_id': 'req-f62726a4-1792-4fa2-b05a-ca6f072175c7'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:48.386911 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f62726a4-1792-4fa2-b05a-ca6f072175c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8cd6af276ec42189f34b4b7f57e3b7a', 'request_id': 'req-f62726a4-1792-4fa2-b05a-ca6f072175c7'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:48.426094 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f62726a4-1792-4fa2-b05a-ca6f072175c7 admin admin] The token cache is being invalidate because project f8cd6af276ec42189f34b4b7f57e3b7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:59:48.426402 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f62726a4-1792-4fa2-b05a-ca6f072175c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:48.430126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f62726a4-1792-4fa2-b05a-ca6f072175c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:48.435290 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 897/1803] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 18:59:47 2022] DELETE /identity/v3/projects/f8cd6af276ec42189f34b4b7f57e3b7a => generated 0 bytes in 842 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:48.436316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62726a4-1792-4fa2-b05a-ca6f072175c7 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:48.444194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f62726a4-1792-4fa2-b05a-ca6f072175c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:48.579780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:48.642685 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62726a4-1792-4fa2-b05a-ca6f072175c7 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:48.663492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b889ed-e452-4d81-b82a-c3c4b3f51947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:48.664126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b889ed-e452-4d81-b82a-c3c4b3f51947 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:48.664383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b889ed-e452-4d81-b82a-c3c4b3f51947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:48.664660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b889ed-e452-4d81-b82a-c3c4b3f51947 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:48.665392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b889ed-e452-4d81-b82a-c3c4b3f51947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:48.667078 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b889ed-e452-4d81-b82a-c3c4b3f51947 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:48.710930 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 898/1804] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:48.712456 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ac6b4-e7c8-4a99-aeb4-55b0acce90ec None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:48.712775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ac6b4-e7c8-4a99-aeb4-55b0acce90ec None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:48.713160 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ac6b4-e7c8-4a99-aeb4-55b0acce90ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:48.773138 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-da8ac6b4-e7c8-4a99-aeb4-55b0acce90ec None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:48.774556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-da8ac6b4-e7c8-4a99-aeb4-55b0acce90ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:48.825551 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 899/1805] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:48.826555 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ac6b4-e7c8-4a99-aeb4-55b0acce90ec None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:48.858971 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ac6b4-e7c8-4a99-aeb4-55b0acce90ec None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:48.890310 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee5e5b-0c62-49e6-a4b6-e9a985f7f312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:48.892139 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee5e5b-0c62-49e6-a4b6-e9a985f7f312 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:48.892705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee5e5b-0c62-49e6-a4b6-e9a985f7f312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:48.892993 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee5e5b-0c62-49e6-a4b6-e9a985f7f312 admin admin] PATH_INFO: `/v3/projects/2245c4a8de4e4d97bf64a27c97c87fac/users/b4ae53815d0242e598686d8ae742ad26/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:48.935649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee5e5b-0c62-49e6-a4b6-e9a985f7f312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2245c4a8de4e4d97bf64a27c97c87fac, user_id=b4ae53815d0242e598686d8ae742ad26, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:48.939457 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee5e5b-0c62-49e6-a4b6-e9a985f7f312 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:48.991081 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 900/1806] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:48 2022] PUT /identity/v3/projects/2245c4a8de4e4d97bf64a27c97c87fac/users/b4ae53815d0242e598686d8ae742ad26/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:48.992769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d8e8e-1cbf-47fe-b1fc-9efadbf3b900 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:48.993060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d8e8e-1cbf-47fe-b1fc-9efadbf3b900 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:48.993409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d8e8e-1cbf-47fe-b1fc-9efadbf3b900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:49.087289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-991d8e8e-1cbf-47fe-b1fc-9efadbf3b900 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:49.088711 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-991d8e8e-1cbf-47fe-b1fc-9efadbf3b900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:49.174877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 907/1807] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Jun 16 18:59:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 1016 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:49.176361 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb841808-5a51-44d0-b60d-f2a57c58d74b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:49.176648 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb841808-5a51-44d0-b60d-f2a57c58d74b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:49.176996 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb841808-5a51-44d0-b60d-f2a57c58d74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:49.339095 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-fb841808-5a51-44d0-b60d-f2a57c58d74b None None] MFA Rules not processed for user `b4ae53815d0242e598686d8ae742ad26`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:49.340827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-fb841808-5a51-44d0-b60d-f2a57c58d74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:49.463603 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 901/1808] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:49.471558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d8e8e-1cbf-47fe-b1fc-9efadbf3b900 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:49.513552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d8e8e-1cbf-47fe-b1fc-9efadbf3b900 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:49.530410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e28fd-6014-4bf2-84c3-83b4851e51f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:49.533359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e28fd-6014-4bf2-84c3-83b4851e51f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:49.533877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e28fd-6014-4bf2-84c3-83b4851e51f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:49.536947 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e28fd-6014-4bf2-84c3-83b4851e51f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:49.538192 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e28fd-6014-4bf2-84c3-83b4851e51f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:59:49.538753 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e28fd-6014-4bf2-84c3-83b4851e51f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:49.585344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e28fd-6014-4bf2-84c3-83b4851e51f3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:49.686145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 902/1809] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:59:49.700815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e28fd-6014-4bf2-84c3-83b4851e51f3 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:49.716700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d81e28fd-6014-4bf2-84c3-83b4851e51f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:49.821098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 908/1810] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 645 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:49.823220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb841808-5a51-44d0-b60d-f2a57c58d74b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:49.888348 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb841808-5a51-44d0-b60d-f2a57c58d74b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:49.912642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0fe98-387a-484c-971a-aefd76e2806f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:49.913950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0fe98-387a-484c-971a-aefd76e2806f admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:49.914369 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0fe98-387a-484c-971a-aefd76e2806f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:49.915152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0fe98-387a-484c-971a-aefd76e2806f admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:49.916426 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0fe98-387a-484c-971a-aefd76e2806f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:49.941848 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0fe98-387a-484c-971a-aefd76e2806f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:49.981481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:50.096044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 909/1811] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:49 2022] POST /identity/v3/projects => generated 381 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:50.098158 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0fe98-387a-484c-971a-aefd76e2806f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:50.124772 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e28fd-6014-4bf2-84c3-83b4851e51f3 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:50.261364 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:50.335207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b97a5d-72b0-425c-baf5-7d3589bffc2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:50.335903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b97a5d-72b0-425c-baf5-7d3589bffc2d service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:50.340561 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b97a5d-72b0-425c-baf5-7d3589bffc2d service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:50.340914 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b97a5d-72b0-425c-baf5-7d3589bffc2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:50.372816 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b97a5d-72b0-425c-baf5-7d3589bffc2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:50.374082 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b97a5d-72b0-425c-baf5-7d3589bffc2d service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:50.390002 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 903/1812] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 703 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:50.390871 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b97a5d-72b0-425c-baf5-7d3589bffc2d service nova] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:50.403525 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0fe98-387a-484c-971a-aefd76e2806f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:50.404088 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b97a5d-72b0-425c-baf5-7d3589bffc2d service nova] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:50.411612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd15153-cc3b-45f9-8ae8-2e0ef68e4b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:50.412301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd15153-cc3b-45f9-8ae8-2e0ef68e4b28 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:50.412576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd15153-cc3b-45f9-8ae8-2e0ef68e4b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:50.412861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd15153-cc3b-45f9-8ae8-2e0ef68e4b28 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:50.414352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd15153-cc3b-45f9-8ae8-2e0ef68e4b28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:50.416009 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd15153-cc3b-45f9-8ae8-2e0ef68e4b28 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:50.424691 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb8c38-69b7-47d2-925a-f306df366856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:50.426252 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb8c38-69b7-47d2-925a-f306df366856 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:50.427042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb8c38-69b7-47d2-925a-f306df366856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:50.427749 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb8c38-69b7-47d2-925a-f306df366856 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:50.429264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb8c38-69b7-47d2-925a-f306df366856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:50.433258 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb8c38-69b7-47d2-925a-f306df366856 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:50.463144 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 904/1813] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:50 2022] POST /identity/v3/users => generated 291 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:50.470120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd15153-cc3b-45f9-8ae8-2e0ef68e4b28 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:50.478260 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-efd15153-cc3b-45f9-8ae8-2e0ef68e4b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:50.511875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd15153-cc3b-45f9-8ae8-2e0ef68e4b28 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:50.536588 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 910/1814] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:50 2022] POST /identity/v3/projects => generated 399 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:50.537090 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb8c38-69b7-47d2-925a-f306df366856 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:50.542766 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50633e34-4138-45c6-af73-372223937df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:50.543916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50633e34-4138-45c6-af73-372223937df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:50.544354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50633e34-4138-45c6-af73-372223937df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:50.545359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50633e34-4138-45c6-af73-372223937df7 admin admin] PATH_INFO: `/v3/projects/8783e2da2acb4c6cb8d81356adf11816` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:50.549798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50633e34-4138-45c6-af73-372223937df7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8783e2da2acb4c6cb8d81356adf11816)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:50.556395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50633e34-4138-45c6-af73-372223937df7 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:50.695548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-50633e34-4138-45c6-af73-372223937df7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8783e2da2acb4c6cb8d81356adf11816', 'request_id': 'req-50633e34-4138-45c6-af73-372223937df7'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:50.710296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb8c38-69b7-47d2-925a-f306df366856 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:50.722425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-50633e34-4138-45c6-af73-372223937df7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8783e2da2acb4c6cb8d81356adf11816', 'request_id': 'req-50633e34-4138-45c6-af73-372223937df7'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:50.732167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-50633e34-4138-45c6-af73-372223937df7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8783e2da2acb4c6cb8d81356adf11816', 'request_id': 'req-50633e34-4138-45c6-af73-372223937df7'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:50.753751 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-50633e34-4138-45c6-af73-372223937df7 admin admin] The token cache is being invalidate because project 8783e2da2acb4c6cb8d81356adf11816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:59:50.754048 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-50633e34-4138-45c6-af73-372223937df7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:50.755642 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-50633e34-4138-45c6-af73-372223937df7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:50.762610 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 905/1815] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 18:59:50 2022] DELETE /identity/v3/projects/8783e2da2acb4c6cb8d81356adf11816 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:50.762610 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50633e34-4138-45c6-af73-372223937df7 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:50.764390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-50633e34-4138-45c6-af73-372223937df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:50.851611 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feebf0f1-0570-40d1-a9a5-e5a227f25595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:50.852381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebf0f1-0570-40d1-a9a5-e5a227f25595 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:50.852665 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebf0f1-0570-40d1-a9a5-e5a227f25595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:50.854819 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebf0f1-0570-40d1-a9a5-e5a227f25595 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:50.854991 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feebf0f1-0570-40d1-a9a5-e5a227f25595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:50.856703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feebf0f1-0570-40d1-a9a5-e5a227f25595 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:50.876488 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 911/1816] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:50.878559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcce3f-e224-4ece-805f-25a02264083d None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:50.878887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcce3f-e224-4ece-805f-25a02264083d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:50.879539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcce3f-e224-4ece-805f-25a02264083d None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:50.882122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 912/1817] 10.222.0.70 () {54 vars in 921 bytes} [Thu Jun 16 18:59:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:59:50.883525 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc8d64-8313-4de1-beeb-fbd15d3742e7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:50.883859 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc8d64-8313-4de1-beeb-fbd15d3742e7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:50.884307 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc8d64-8313-4de1-beeb-fbd15d3742e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:50.908192 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50633e34-4138-45c6-af73-372223937df7 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:50.939084 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d403d19-bb02-4962-9297-5d0f4a95677c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:50.941694 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d403d19-bb02-4962-9297-5d0f4a95677c admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:50.941986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d403d19-bb02-4962-9297-5d0f4a95677c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:50.942407 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d403d19-bb02-4962-9297-5d0f4a95677c admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:50.951017 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d403d19-bb02-4962-9297-5d0f4a95677c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:50.959240 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d403d19-bb02-4962-9297-5d0f4a95677c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:50.988880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-44cc8d64-8313-4de1-beeb-fbd15d3742e7 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:50.993178 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-44cc8d64-8313-4de1-beeb-fbd15d3742e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:51.012698 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 913/1818] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:51.013941 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc8d64-8313-4de1-beeb-fbd15d3742e7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:51.017718 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-44cc8d64-8313-4de1-beeb-fbd15d3742e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:51.040705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 906/1819] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:50 2022] POST /identity/v3/users => generated 306 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:51.046311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc2a15-877f-4fb4-8f0d-cb7464434e59 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:51.046704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc2a15-877f-4fb4-8f0d-cb7464434e59 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:51.048961 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc2a15-877f-4fb4-8f0d-cb7464434e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:51.052490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc8d64-8313-4de1-beeb-fbd15d3742e7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:51.067461 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596655b-21c7-4af7-85ad-6d703fcdc303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:51.068224 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596655b-21c7-4af7-85ad-6d703fcdc303 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:51.068564 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596655b-21c7-4af7-85ad-6d703fcdc303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:51.068840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596655b-21c7-4af7-85ad-6d703fcdc303 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:51.070195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596655b-21c7-4af7-85ad-6d703fcdc303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:51.082743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596655b-21c7-4af7-85ad-6d703fcdc303 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:51.086083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:51.102718 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 914/1820] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:51.103081 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-88dc2a15-877f-4fb4-8f0d-cb7464434e59 None None] MFA Rules not processed for user `8342da838fe4462b9de2817ad3ac4ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:51.103469 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596655b-21c7-4af7-85ad-6d703fcdc303 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:51.105119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-88dc2a15-877f-4fb4-8f0d-cb7464434e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:51.106051 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5596655b-21c7-4af7-85ad-6d703fcdc303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:51.119498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:51.137486 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596655b-21c7-4af7-85ad-6d703fcdc303 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:51.157125 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b39bf-bc52-431f-8966-ae1972e4e8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:51.158156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b39bf-bc52-431f-8966-ae1972e4e8bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:51.158447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b39bf-bc52-431f-8966-ae1972e4e8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:51.158884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b39bf-bc52-431f-8966-ae1972e4e8bd admin admin] PATH_INFO: `/v3/projects/d1d433a6ea254dc8844b9cf736fee3f0` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:51.161695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b39bf-bc52-431f-8966-ae1972e4e8bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d433a6ea254dc8844b9cf736fee3f0)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:51.165146 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b39bf-bc52-431f-8966-ae1972e4e8bd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:51.234124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 907/1821] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Jun 16 18:59:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:51.236459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f43842-182c-4b5e-8777-d787cace5411 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:51.236818 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f43842-182c-4b5e-8777-d787cace5411 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:51.237343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f43842-182c-4b5e-8777-d787cace5411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:51.274725 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-a1f43842-182c-4b5e-8777-d787cace5411 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:51.276626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a1f43842-182c-4b5e-8777-d787cace5411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:51.301460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f84b39bf-bc52-431f-8966-ae1972e4e8bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d433a6ea254dc8844b9cf736fee3f0', 'request_id': 'req-f84b39bf-bc52-431f-8966-ae1972e4e8bd'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:51.321536 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f84b39bf-bc52-431f-8966-ae1972e4e8bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d433a6ea254dc8844b9cf736fee3f0', 'request_id': 'req-f84b39bf-bc52-431f-8966-ae1972e4e8bd'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:51.336195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f84b39bf-bc52-431f-8966-ae1972e4e8bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d433a6ea254dc8844b9cf736fee3f0', 'request_id': 'req-f84b39bf-bc52-431f-8966-ae1972e4e8bd'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:51.356761 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 908/1822] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:51.358008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f43842-182c-4b5e-8777-d787cace5411 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:51.361894 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f84b39bf-bc52-431f-8966-ae1972e4e8bd admin admin] The token cache is being invalidate because project d1d433a6ea254dc8844b9cf736fee3f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:59:51.362348 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f84b39bf-bc52-431f-8966-ae1972e4e8bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:51.363586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f84b39bf-bc52-431f-8966-ae1972e4e8bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:51.366548 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 915/1823] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 18:59:51 2022] DELETE /identity/v3/projects/d1d433a6ea254dc8844b9cf736fee3f0 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:51.372404 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b39bf-bc52-431f-8966-ae1972e4e8bd admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:51.374511 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f84b39bf-bc52-431f-8966-ae1972e4e8bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:51.375555 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f43842-182c-4b5e-8777-d787cace5411 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:51.381727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3869958-f057-4e52-9a5d-18a6002ae7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:51.382921 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3869958-f057-4e52-9a5d-18a6002ae7e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:51.383411 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3869958-f057-4e52-9a5d-18a6002ae7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:51.383947 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3869958-f057-4e52-9a5d-18a6002ae7e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:51.385111 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3869958-f057-4e52-9a5d-18a6002ae7e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:51.388840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3869958-f057-4e52-9a5d-18a6002ae7e2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:51.402600 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b39bf-bc52-431f-8966-ae1972e4e8bd admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:51.409588 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 909/1824] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:51.410957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3869958-f057-4e52-9a5d-18a6002ae7e2 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:51.412763 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969e413-9cf6-4a81-b2a3-288cd96fc766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:51.413874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969e413-9cf6-4a81-b2a3-288cd96fc766 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:51.414252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d3869958-f057-4e52-9a5d-18a6002ae7e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:51.414370 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969e413-9cf6-4a81-b2a3-288cd96fc766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:51.414741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969e413-9cf6-4a81-b2a3-288cd96fc766 admin admin] PATH_INFO: `/v3/projects/5059d65bb0674eb2bb799d3140e9c719/users/3a686db6e03b45f8b2813c33b35ec123/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:51.446050 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969e413-9cf6-4a81-b2a3-288cd96fc766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5059d65bb0674eb2bb799d3140e9c719, user_id=3a686db6e03b45f8b2813c33b35ec123, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:51.451508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969e413-9cf6-4a81-b2a3-288cd96fc766 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:51.479125 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 916/1825] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:51 2022] PUT /identity/v3/projects/5059d65bb0674eb2bb799d3140e9c719/users/3a686db6e03b45f8b2813c33b35ec123/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:51.480132 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969e413-9cf6-4a81-b2a3-288cd96fc766 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:51.482317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c969e413-9cf6-4a81-b2a3-288cd96fc766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:51.535720 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3869958-f057-4e52-9a5d-18a6002ae7e2 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:51.585278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:51.607097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969e413-9cf6-4a81-b2a3-288cd96fc766 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:51.620005 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351feda-8d12-4d0e-8c26-cdf4ba615c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:51.621200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351feda-8d12-4d0e-8c26-cdf4ba615c8a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:51.621908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351feda-8d12-4d0e-8c26-cdf4ba615c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:51.622363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351feda-8d12-4d0e-8c26-cdf4ba615c8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:51.623956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351feda-8d12-4d0e-8c26-cdf4ba615c8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:59:51.624438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351feda-8d12-4d0e-8c26-cdf4ba615c8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:51.638448 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351feda-8d12-4d0e-8c26-cdf4ba615c8a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:51.653502 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d3454-9322-4071-aa6b-94f668a1d6f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:51.654271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d3454-9322-4071-aa6b-94f668a1d6f9 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:51.654661 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d3454-9322-4071-aa6b-94f668a1d6f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:51.655173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d3454-9322-4071-aa6b-94f668a1d6f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:51.663277 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 917/1826] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:59:51.663485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-813d3454-9322-4071-aa6b-94f668a1d6f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:51.663603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351feda-8d12-4d0e-8c26-cdf4ba615c8a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:51.663603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b351feda-8d12-4d0e-8c26-cdf4ba615c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:51.677025 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d3454-9322-4071-aa6b-94f668a1d6f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:51.678834 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d3454-9322-4071-aa6b-94f668a1d6f9 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:51.685917 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351feda-8d12-4d0e-8c26-cdf4ba615c8a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:51.695293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a64e1e-0398-4d79-8374-a0e2493c7703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:51.696845 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a64e1e-0398-4d79-8374-a0e2493c7703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:51.697358 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a64e1e-0398-4d79-8374-a0e2493c7703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:51.697866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a64e1e-0398-4d79-8374-a0e2493c7703 admin admin] PATH_INFO: `/v3/users/864b638582984f8d82e46901c106e65c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:51.700795 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a64e1e-0398-4d79-8374-a0e2493c7703 admin admin] RBAC: Authorizing `identity:delete_user(user_id=864b638582984f8d82e46901c106e65c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:51.705134 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a64e1e-0398-4d79-8374-a0e2493c7703 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:51.705785 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 910/1827] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:51.707394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d3454-9322-4071-aa6b-94f668a1d6f9 service nova] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:51.712987 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-813d3454-9322-4071-aa6b-94f668a1d6f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:51.751001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d3454-9322-4071-aa6b-94f668a1d6f9 service nova] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:51.761018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac667ef-7888-4ab3-b6cf-0c15b07b160c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:51.763446 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac667ef-7888-4ab3-b6cf-0c15b07b160c admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:51.763978 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac667ef-7888-4ab3-b6cf-0c15b07b160c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:51.764432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac667ef-7888-4ab3-b6cf-0c15b07b160c admin admin] PATH_INFO: `/v3/projects/8f6280684244433cb081e48d86f3c02b/users/30dea9def7a242ab8d2b6efc275f9da5/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:51.791574 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac667ef-7888-4ab3-b6cf-0c15b07b160c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6280684244433cb081e48d86f3c02b, user_id=30dea9def7a242ab8d2b6efc275f9da5, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:51.794893 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:51.797960 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac667ef-7888-4ab3-b6cf-0c15b07b160c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:51.798820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b2a64e1e-0398-4d79-8374-a0e2493c7703 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '864b638582984f8d82e46901c106e65c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:51.800059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b2a64e1e-0398-4d79-8374-a0e2493c7703 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '864b638582984f8d82e46901c106e65c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:51.801395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b2a64e1e-0398-4d79-8374-a0e2493c7703 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '864b638582984f8d82e46901c106e65c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:51.813916 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b2a64e1e-0398-4d79-8374-a0e2493c7703 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '864b638582984f8d82e46901c106e65c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:51.822975 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b2a64e1e-0398-4d79-8374-a0e2493c7703 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '864b638582984f8d82e46901c106e65c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:51.836520 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 911/1828] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:51 2022] PUT /identity/v3/projects/8f6280684244433cb081e48d86f3c02b/users/30dea9def7a242ab8d2b6efc275f9da5/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:51.838963 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c0101-7a47-4fc5-b171-4d4e533f0c34 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:51.839730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c0101-7a47-4fc5-b171-4d4e533f0c34 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:51.840376 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c0101-7a47-4fc5-b171-4d4e533f0c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:51.844901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 918/1829] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:51 2022] DELETE /identity/v3/users/864b638582984f8d82e46901c106e65c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:51.845693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a64e1e-0398-4d79-8374-a0e2493c7703 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:51.847122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b2a64e1e-0398-4d79-8374-a0e2493c7703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:51.883359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-171c0101-7a47-4fc5-b171-4d4e533f0c34 None None] MFA Rules not processed for user `3a686db6e03b45f8b2813c33b35ec123`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:51.884789 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-171c0101-7a47-4fc5-b171-4d4e533f0c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:51.955044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a64e1e-0398-4d79-8374-a0e2493c7703 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:51.965479 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678440fe-9e86-4a79-b1d8-5192b9dd8f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:51.966238 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678440fe-9e86-4a79-b1d8-5192b9dd8f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:51.966505 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678440fe-9e86-4a79-b1d8-5192b9dd8f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:51.966782 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678440fe-9e86-4a79-b1d8-5192b9dd8f2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:51.967509 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678440fe-9e86-4a79-b1d8-5192b9dd8f2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:51.969957 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678440fe-9e86-4a79-b1d8-5192b9dd8f2b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:51.988375 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 912/1830] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:51.988375 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c0101-7a47-4fc5-b171-4d4e533f0c34 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:51.988375 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-171c0101-7a47-4fc5-b171-4d4e533f0c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:52.015202 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 919/1831] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:51 2022] POST /identity/v3/projects => generated 375 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:52.015202 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678440fe-9e86-4a79-b1d8-5192b9dd8f2b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:52.015202 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-678440fe-9e86-4a79-b1d8-5192b9dd8f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:52.017952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c0101-7a47-4fc5-b171-4d4e533f0c34 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:52.086809 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5dddc1-4cb8-45e6-a6cd-c8d6e14bee35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:52.087884 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5dddc1-4cb8-45e6-a6cd-c8d6e14bee35 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:52.088401 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5dddc1-4cb8-45e6-a6cd-c8d6e14bee35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:52.088879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5dddc1-4cb8-45e6-a6cd-c8d6e14bee35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:52.089874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5dddc1-4cb8-45e6-a6cd-c8d6e14bee35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:52.091746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5dddc1-4cb8-45e6-a6cd-c8d6e14bee35 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:52.108560 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 913/1832] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:52.108770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678440fe-9e86-4a79-b1d8-5192b9dd8f2b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:52.108859 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5dddc1-4cb8-45e6-a6cd-c8d6e14bee35 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:52.115638 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37080b-936a-4d1e-aabf-f834aa405c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:52.116973 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37080b-936a-4d1e-aabf-f834aa405c23 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:52.117414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37080b-936a-4d1e-aabf-f834aa405c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:52.117887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37080b-936a-4d1e-aabf-f834aa405c23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:52.118929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37080b-936a-4d1e-aabf-f834aa405c23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:52.121228 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5dddc1-4cb8-45e6-a6cd-c8d6e14bee35 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:52.127756 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbe437-82f1-45aa-a2d4-f6f71c278262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:52.128246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37080b-936a-4d1e-aabf-f834aa405c23 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:52.128355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbe437-82f1-45aa-a2d4-f6f71c278262 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:52.128633 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbe437-82f1-45aa-a2d4-f6f71c278262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:52.128908 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbe437-82f1-45aa-a2d4-f6f71c278262 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:52.129806 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbe437-82f1-45aa-a2d4-f6f71c278262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:52.131790 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbe437-82f1-45aa-a2d4-f6f71c278262 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:52.165267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 914/1833] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:52 2022] POST /identity/v3/users => generated 288 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:52.166754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbe437-82f1-45aa-a2d4-f6f71c278262 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:52.168735 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 920/1834] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:52 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:52.169785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37080b-936a-4d1e-aabf-f834aa405c23 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:52.176477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:52.186995 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbe437-82f1-45aa-a2d4-f6f71c278262 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:52.215182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:52.265462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37080b-936a-4d1e-aabf-f834aa405c23 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:52.279111 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3348f4b-dfd0-46e7-a36d-ee0894257746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:52.281003 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348f4b-dfd0-46e7-a36d-ee0894257746 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:52.282605 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348f4b-dfd0-46e7-a36d-ee0894257746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:52.283436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348f4b-dfd0-46e7-a36d-ee0894257746 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:52.285745 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3348f4b-dfd0-46e7-a36d-ee0894257746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:52.291893 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3348f4b-dfd0-46e7-a36d-ee0894257746 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:52.319307 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c2c0d-5e15-48c9-b04a-a7e5878c37aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:52.319769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c2c0d-5e15-48c9-b04a-a7e5878c37aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:52.319914 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c2c0d-5e15-48c9-b04a-a7e5878c37aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:52.320194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c2c0d-5e15-48c9-b04a-a7e5878c37aa admin admin] PATH_INFO: `/v3/projects/8f6280684244433cb081e48d86f3c02b/users/30dea9def7a242ab8d2b6efc275f9da5/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:52.324523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c2c0d-5e15-48c9-b04a-a7e5878c37aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6280684244433cb081e48d86f3c02b, user_id=30dea9def7a242ab8d2b6efc275f9da5, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:52.327403 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c2c0d-5e15-48c9-b04a-a7e5878c37aa admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:52.341720 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 921/1835] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:52.342846 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3348f4b-dfd0-46e7-a36d-ee0894257746 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:52.365759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3348f4b-dfd0-46e7-a36d-ee0894257746 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:52.397629 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 915/1836] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:52 2022] PUT /identity/v3/projects/8f6280684244433cb081e48d86f3c02b/users/30dea9def7a242ab8d2b6efc275f9da5/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:52.398819 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c2c0d-5e15-48c9-b04a-a7e5878c37aa admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:52.497400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33b88e-e4df-4baa-8f7f-af6d09110224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:52.498188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33b88e-e4df-4baa-8f7f-af6d09110224 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:52.499309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33b88e-e4df-4baa-8f7f-af6d09110224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:52.499309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33b88e-e4df-4baa-8f7f-af6d09110224 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:52.500550 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33b88e-e4df-4baa-8f7f-af6d09110224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:52.503516 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33b88e-e4df-4baa-8f7f-af6d09110224 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:52.520666 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c2c0d-5e15-48c9-b04a-a7e5878c37aa admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:52.529836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644b8b3-02c4-4d7c-9635-43089e4073af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:52.530669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644b8b3-02c4-4d7c-9635-43089e4073af admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:52.530961 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644b8b3-02c4-4d7c-9635-43089e4073af admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:52.531518 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644b8b3-02c4-4d7c-9635-43089e4073af admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:52.533160 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644b8b3-02c4-4d7c-9635-43089e4073af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:52.536583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644b8b3-02c4-4d7c-9635-43089e4073af admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:52.537796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 922/1837] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:52 2022] POST /identity/v3/users => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:52.539713 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da26e57-c4e4-4e89-8f97-c9141d1a4c2f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:52.540204 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da26e57-c4e4-4e89-8f97-c9141d1a4c2f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:52.540665 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da26e57-c4e4-4e89-8f97-c9141d1a4c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:52.552883 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 916/1838] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:52.553969 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644b8b3-02c4-4d7c-9635-43089e4073af admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:52.556074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e644b8b3-02c4-4d7c-9635-43089e4073af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:52.578825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644b8b3-02c4-4d7c-9635-43089e4073af admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:52.581365 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8da26e57-c4e4-4e89-8f97-c9141d1a4c2f None None] MFA Rules not processed for user `30dea9def7a242ab8d2b6efc275f9da5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:52.582948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8da26e57-c4e4-4e89-8f97-c9141d1a4c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:52.607332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:52.608252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:52.608486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:52.608787 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0 admin admin] PATH_INFO: `/v3/projects/b4e33caa7ba84f0586fd544b6335e7b8` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:52.612310 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4e33caa7ba84f0586fd544b6335e7b8)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:52.617936 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:52.687364 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 923/1839] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:52 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:52.692431 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da26e57-c4e4-4e89-8f97-c9141d1a4c2f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:52.707842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da26e57-c4e4-4e89-8f97-c9141d1a4c2f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:52.712009 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4e33caa7ba84f0586fd544b6335e7b8', 'request_id': 'req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:52.726740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4e33caa7ba84f0586fd544b6335e7b8', 'request_id': 'req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:52.739077 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4e33caa7ba84f0586fd544b6335e7b8', 'request_id': 'req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:52.772470 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0 admin admin] The token cache is being invalidate because project b4e33caa7ba84f0586fd544b6335e7b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:59:52.773861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:52.777436 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:52.790857 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 917/1840] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 18:59:52 2022] DELETE /identity/v3/projects/b4e33caa7ba84f0586fd544b6335e7b8 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:52.792290 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:52.795866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:52.804653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:52.841916 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af431b38-c93d-4c12-8959-952d9811325b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:52.843035 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af431b38-c93d-4c12-8959-952d9811325b admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:52.843559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af431b38-c93d-4c12-8959-952d9811325b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:52.845194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af431b38-c93d-4c12-8959-952d9811325b admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:52.845409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:52.847728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af431b38-c93d-4c12-8959-952d9811325b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:52.849900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af431b38-c93d-4c12-8959-952d9811325b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:52.879159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 924/1841] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:52.880787 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af431b38-c93d-4c12-8959-952d9811325b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:52.882901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-af431b38-c93d-4c12-8959-952d9811325b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:52.949394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b377b7c-08db-42f1-b39d-5c9297fe7eb0 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:52.960492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81957cd7-2305-46fe-a1a1-d4059e187138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:52.960492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81957cd7-2305-46fe-a1a1-d4059e187138 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:52.960492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81957cd7-2305-46fe-a1a1-d4059e187138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:52.960492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81957cd7-2305-46fe-a1a1-d4059e187138 admin admin] PATH_INFO: `/v3/projects/de4b8df58fd64508a56495dfa6d5c63d/users/6a75fba2951a465ca3f064f654c8ae51/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:52.980468 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af431b38-c93d-4c12-8959-952d9811325b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:52.985139 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81957cd7-2305-46fe-a1a1-d4059e187138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de4b8df58fd64508a56495dfa6d5c63d, user_id=6a75fba2951a465ca3f064f654c8ae51, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:52.985672 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ea84a-4d69-4b6b-bbc9-cae50ee14926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:52.986638 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ea84a-4d69-4b6b-bbc9-cae50ee14926 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:52.987140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ea84a-4d69-4b6b-bbc9-cae50ee14926 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:52.987627 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ea84a-4d69-4b6b-bbc9-cae50ee14926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:52.988328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81957cd7-2305-46fe-a1a1-d4059e187138 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:52.990075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-592ea84a-4d69-4b6b-bbc9-cae50ee14926 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:53.012935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 918/1842] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:52 2022] PUT /identity/v3/projects/de4b8df58fd64508a56495dfa6d5c63d/users/6a75fba2951a465ca3f064f654c8ae51/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:53.015411 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341ee35-bd13-4b8f-ad12-dfac01a307c1 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:53.016056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341ee35-bd13-4b8f-ad12-dfac01a307c1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:53.018135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341ee35-bd13-4b8f-ad12-dfac01a307c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:53.057224 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c341ee35-bd13-4b8f-ad12-dfac01a307c1 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:53.058920 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c341ee35-bd13-4b8f-ad12-dfac01a307c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:53.112659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ea84a-4d69-4b6b-bbc9-cae50ee14926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:53.114129 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ea84a-4d69-4b6b-bbc9-cae50ee14926 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:53.145183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 925/1843] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:52 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:53.146320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ea84a-4d69-4b6b-bbc9-cae50ee14926 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:53.149992 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-592ea84a-4d69-4b6b-bbc9-cae50ee14926 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:53.176706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ea84a-4d69-4b6b-bbc9-cae50ee14926 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:53.177334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 919/1844] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:53.190746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea016e1-9a53-47f4-9cd7-7d70742693de None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:53.191304 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea016e1-9a53-47f4-9cd7-7d70742693de None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:53.191687 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea016e1-9a53-47f4-9cd7-7d70742693de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:53.213405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f656d-0fb8-49ff-9757-f92909a96604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:53.214542 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f656d-0fb8-49ff-9757-f92909a96604 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:53.216595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f656d-0fb8-49ff-9757-f92909a96604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:53.217124 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f656d-0fb8-49ff-9757-f92909a96604 admin admin] PATH_INFO: `/v3/projects/f0d5c17586014530a8c21865450ea71e/users/8485c6ad1ec7445db034a005ea18da02/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:53.247903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-fea016e1-9a53-47f4-9cd7-7d70742693de None None] MFA Rules not processed for user `6a75fba2951a465ca3f064f654c8ae51`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:53.249676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-fea016e1-9a53-47f4-9cd7-7d70742693de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:53.278186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f656d-0fb8-49ff-9757-f92909a96604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0d5c17586014530a8c21865450ea71e, user_id=8485c6ad1ec7445db034a005ea18da02, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:53.292558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f656d-0fb8-49ff-9757-f92909a96604 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:53.323292 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 926/1845] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:53 2022] PUT /identity/v3/projects/f0d5c17586014530a8c21865450ea71e/users/8485c6ad1ec7445db034a005ea18da02/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:53.325328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8b6f3-7c3b-474c-9ae2-dd825846dcea None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:53.326185 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8b6f3-7c3b-474c-9ae2-dd825846dcea None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:53.328653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8b6f3-7c3b-474c-9ae2-dd825846dcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:53.358463 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2bd8b6f3-7c3b-474c-9ae2-dd825846dcea None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:53.361912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2bd8b6f3-7c3b-474c-9ae2-dd825846dcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:53.394083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 920/1846] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:53.396845 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea016e1-9a53-47f4-9cd7-7d70742693de None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:53.458368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 927/1847] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:53.459671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8b6f3-7c3b-474c-9ae2-dd825846dcea None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:53.471564 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8b6f3-7c3b-474c-9ae2-dd825846dcea None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:53.477877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6d6fc-6728-4bd3-8003-cffd9a13198a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:53.478965 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6d6fc-6728-4bd3-8003-cffd9a13198a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:53.479939 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6d6fc-6728-4bd3-8003-cffd9a13198a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:53.480453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6d6fc-6728-4bd3-8003-cffd9a13198a admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:53.481447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6d6fc-6728-4bd3-8003-cffd9a13198a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:53.483339 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6d6fc-6728-4bd3-8003-cffd9a13198a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:53.486755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:53.509039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:53.511172 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 928/1848] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:53.512058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6d6fc-6728-4bd3-8003-cffd9a13198a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:53.548419 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6d6fc-6728-4bd3-8003-cffd9a13198a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:53.558032 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5e2d7-ff82-433e-a76f-d9e5c08d627f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:53.559229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5e2d7-ff82-433e-a76f-d9e5c08d627f admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:53.559515 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5e2d7-ff82-433e-a76f-d9e5c08d627f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:53.559849 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5e2d7-ff82-433e-a76f-d9e5c08d627f admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:53.561724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea016e1-9a53-47f4-9cd7-7d70742693de None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:53.562303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5e2d7-ff82-433e-a76f-d9e5c08d627f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:53.564981 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5e2d7-ff82-433e-a76f-d9e5c08d627f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:53.620805 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 929/1849] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:53 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:53.621676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5e2d7-ff82-433e-a76f-d9e5c08d627f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:53.636483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e69e8-5300-42ae-8813-fe4b786e9266 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:53.637933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e69e8-5300-42ae-8813-fe4b786e9266 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:53.638228 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e69e8-5300-42ae-8813-fe4b786e9266 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:53.638563 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e69e8-5300-42ae-8813-fe4b786e9266 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:53.642939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1f1e69e8-5300-42ae-8813-fe4b786e9266 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:53.745825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e69e8-5300-42ae-8813-fe4b786e9266 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:53.747805 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e69e8-5300-42ae-8813-fe4b786e9266 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:53.760002 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 921/1850] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:53.761082 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e69e8-5300-42ae-8813-fe4b786e9266 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:53.761198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5e2d7-ff82-433e-a76f-d9e5c08d627f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:53.767875 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26e635-201a-4399-b3a5-ae0614b3be7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:53.768704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26e635-201a-4399-b3a5-ae0614b3be7f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:53.768987 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26e635-201a-4399-b3a5-ae0614b3be7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:53.769261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26e635-201a-4399-b3a5-ae0614b3be7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:53.771491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26e635-201a-4399-b3a5-ae0614b3be7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:59:53.773250 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26e635-201a-4399-b3a5-ae0614b3be7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:53.777268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26e635-201a-4399-b3a5-ae0614b3be7f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:53.782093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e69e8-5300-42ae-8813-fe4b786e9266 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:53.797760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e637cda-1808-4445-976b-00a1aa093cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:53.800945 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 930/1851] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 18:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:59:53.807833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e637cda-1808-4445-976b-00a1aa093cdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:53.807833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e637cda-1808-4445-976b-00a1aa093cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:53.808624 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26e635-201a-4399-b3a5-ae0614b3be7f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:53.810392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e637cda-1808-4445-976b-00a1aa093cdb admin admin] PATH_INFO: `/v3/projects/f0d5c17586014530a8c21865450ea71e/users/8485c6ad1ec7445db034a005ea18da02/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:53.817781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e637cda-1808-4445-976b-00a1aa093cdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0d5c17586014530a8c21865450ea71e, user_id=8485c6ad1ec7445db034a005ea18da02, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:53.830593 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e637cda-1808-4445-976b-00a1aa093cdb admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:53.847686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26e635-201a-4399-b3a5-ae0614b3be7f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:53.861298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031a497-cc2d-4c8b-82fd-82ed75acda26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:53.862305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031a497-cc2d-4c8b-82fd-82ed75acda26 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:53.862827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031a497-cc2d-4c8b-82fd-82ed75acda26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:53.863216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031a497-cc2d-4c8b-82fd-82ed75acda26 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:53.864039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031a497-cc2d-4c8b-82fd-82ed75acda26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:53.866415 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 922/1852] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:53 2022] PUT /identity/v3/projects/f0d5c17586014530a8c21865450ea71e/users/8485c6ad1ec7445db034a005ea18da02/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:53.867445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e637cda-1808-4445-976b-00a1aa093cdb admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:53.877885 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031a497-cc2d-4c8b-82fd-82ed75acda26 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:53.939528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 931/1853] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:53 2022] POST /identity/v3/users => generated 294 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:53.941794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39118417-39c1-4114-9452-967528fcd01c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:53.951152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39118417-39c1-4114-9452-967528fcd01c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:53.956036 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39118417-39c1-4114-9452-967528fcd01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:54.006835 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-39118417-39c1-4114-9452-967528fcd01c None None] MFA Rules not processed for user `8485c6ad1ec7445db034a005ea18da02`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:54.008978 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-39118417-39c1-4114-9452-967528fcd01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:54.021436 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e637cda-1808-4445-976b-00a1aa093cdb admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:54.032845 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7317d2c-fcaf-4288-af57-7d9ce19cd8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:54.034085 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7317d2c-fcaf-4288-af57-7d9ce19cd8ec admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:54.034913 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7317d2c-fcaf-4288-af57-7d9ce19cd8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:54.035414 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7317d2c-fcaf-4288-af57-7d9ce19cd8ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:54.038761 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7317d2c-fcaf-4288-af57-7d9ce19cd8ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:54.045973 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7317d2c-fcaf-4288-af57-7d9ce19cd8ec admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:54.106542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 923/1854] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:53 2022] POST /identity/v3/projects => generated 377 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:54.107733 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7317d2c-fcaf-4288-af57-7d9ce19cd8ec admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:54.124362 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:54.223405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 932/1855] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:54.224213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39118417-39c1-4114-9452-967528fcd01c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:54.250317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39118417-39c1-4114-9452-967528fcd01c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:54.260999 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7317d2c-fcaf-4288-af57-7d9ce19cd8ec admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:54.261842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a76f1d-9044-4035-b888-7f181b924bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:54.263384 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a76f1d-9044-4035-b888-7f181b924bed admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:54.264669 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a76f1d-9044-4035-b888-7f181b924bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:54.267287 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a76f1d-9044-4035-b888-7f181b924bed admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:54.268080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a76f1d-9044-4035-b888-7f181b924bed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:54.270775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a76f1d-9044-4035-b888-7f181b924bed admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:54.277938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277761d1-deb3-4491-9771-24afaa39ab8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:54.278664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277761d1-deb3-4491-9771-24afaa39ab8c admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:54.278954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277761d1-deb3-4491-9771-24afaa39ab8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:54.279374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277761d1-deb3-4491-9771-24afaa39ab8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:54.280117 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277761d1-deb3-4491-9771-24afaa39ab8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:54.282106 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277761d1-deb3-4491-9771-24afaa39ab8c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:54.321694 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 924/1856] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:54.327461 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 933/1857] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:54 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:54.332361 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277761d1-deb3-4491-9771-24afaa39ab8c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:54.333704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a76f1d-9044-4035-b888-7f181b924bed admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:54.362438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a76f1d-9044-4035-b888-7f181b924bed admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:54.385812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a428e45-ff90-438a-9169-d4095d4fae8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:54.397211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a428e45-ff90-438a-9169-d4095d4fae8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:54.400207 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a428e45-ff90-438a-9169-d4095d4fae8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:54.400629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a428e45-ff90-438a-9169-d4095d4fae8e admin admin] PATH_INFO: `/v3/projects/696a9a0ab1964ae5bc5f28ae777b5255/users/c0767eda97af49e2aac1e1f6c33ecc96/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:54.457795 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a428e45-ff90-438a-9169-d4095d4fae8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=696a9a0ab1964ae5bc5f28ae777b5255, user_id=c0767eda97af49e2aac1e1f6c33ecc96, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:54.462083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a428e45-ff90-438a-9169-d4095d4fae8e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:54.465183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:54.500805 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 934/1858] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:54 2022] PUT /identity/v3/projects/696a9a0ab1964ae5bc5f28ae777b5255/users/c0767eda97af49e2aac1e1f6c33ecc96/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:54.501822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a428e45-ff90-438a-9169-d4095d4fae8e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:54.533475 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277761d1-deb3-4491-9771-24afaa39ab8c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:54.652367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a428e45-ff90-438a-9169-d4095d4fae8e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:54.664127 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99e4d6-ac10-4f57-9b1b-3dc6adcc6e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:54.665523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99e4d6-ac10-4f57-9b1b-3dc6adcc6e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:54.666220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99e4d6-ac10-4f57-9b1b-3dc6adcc6e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:54.666965 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99e4d6-ac10-4f57-9b1b-3dc6adcc6e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:54.668620 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99e4d6-ac10-4f57-9b1b-3dc6adcc6e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:54.681573 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99e4d6-ac10-4f57-9b1b-3dc6adcc6e7d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:54.713529 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c6197-0534-48fe-b8fc-58256679950e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:54.714508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c6197-0534-48fe-b8fc-58256679950e service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:54.715331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c6197-0534-48fe-b8fc-58256679950e service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:54.715331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c6197-0534-48fe-b8fc-58256679950e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:54.725710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 935/1859] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:54.727995 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99e4d6-ac10-4f57-9b1b-3dc6adcc6e7d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:54.754591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99e4d6-ac10-4f57-9b1b-3dc6adcc6e7d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:54.775510 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c200b-3a8e-4753-b1cc-9e0d479b3133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:54.781231 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c200b-3a8e-4753-b1cc-9e0d479b3133 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:54.783984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c200b-3a8e-4753-b1cc-9e0d479b3133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:54.786966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c200b-3a8e-4753-b1cc-9e0d479b3133 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:54.789592 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c200b-3a8e-4753-b1cc-9e0d479b3133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:54.799197 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c200b-3a8e-4753-b1cc-9e0d479b3133 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:54.833299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 936/1860] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:54.835965 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c200b-3a8e-4753-b1cc-9e0d479b3133 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:54.882121 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c200b-3a8e-4753-b1cc-9e0d479b3133 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:54.898795 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff667fdc-01fc-44d1-a465-b36341265654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:54.900624 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff667fdc-01fc-44d1-a465-b36341265654 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:54.901739 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff667fdc-01fc-44d1-a465-b36341265654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:54.902965 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff667fdc-01fc-44d1-a465-b36341265654 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:54.904615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c6197-0534-48fe-b8fc-58256679950e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:54.905588 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff667fdc-01fc-44d1-a465-b36341265654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:54.910562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff667fdc-01fc-44d1-a465-b36341265654 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:54.927178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c6197-0534-48fe-b8fc-58256679950e service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:54.970220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 937/1861] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:54.972026 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff667fdc-01fc-44d1-a465-b36341265654 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:54.972861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 925/1862] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:54.974439 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c6197-0534-48fe-b8fc-58256679950e service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:55.010757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff667fdc-01fc-44d1-a465-b36341265654 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:55.010885 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c6197-0534-48fe-b8fc-58256679950e service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:55.026624 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2fedd-b3a4-4666-a511-34682bfb0822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:55.028779 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2311d-fbd0-492d-86b4-e77485019b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:55.030190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2311d-fbd0-492d-86b4-e77485019b88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:55.030669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2311d-fbd0-492d-86b4-e77485019b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:55.031847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2311d-fbd0-492d-86b4-e77485019b88 admin admin] PATH_INFO: `/v3/projects/36ac94e0b9214eb0b0588be9ea5b4c29/users/f3b60e939fb24f6fbbb08c02bb984f9d/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:55.034825 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2fedd-b3a4-4666-a511-34682bfb0822 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:55.035286 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2fedd-b3a4-4666-a511-34682bfb0822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:55.035777 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2fedd-b3a4-4666-a511-34682bfb0822 admin admin] PATH_INFO: `/v3/projects/696a9a0ab1964ae5bc5f28ae777b5255/users/c0767eda97af49e2aac1e1f6c33ecc96/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:55.039327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2fedd-b3a4-4666-a511-34682bfb0822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=696a9a0ab1964ae5bc5f28ae777b5255, user_id=c0767eda97af49e2aac1e1f6c33ecc96, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:55.049637 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2fedd-b3a4-4666-a511-34682bfb0822 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:55.074703 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:55.081483 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 938/1863] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:54 2022] PUT /identity/v3/projects/696a9a0ab1964ae5bc5f28ae777b5255/users/c0767eda97af49e2aac1e1f6c33ecc96/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:55.081757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2311d-fbd0-492d-86b4-e77485019b88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ac94e0b9214eb0b0588be9ea5b4c29, user_id=f3b60e939fb24f6fbbb08c02bb984f9d, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:55.087828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2311d-fbd0-492d-86b4-e77485019b88 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:55.100641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecc833-1f1e-4927-a8d4-5031a3ebbc2f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:55.100936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecc833-1f1e-4927-a8d4-5031a3ebbc2f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:55.101298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecc833-1f1e-4927-a8d4-5031a3ebbc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:55.119054 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 926/1864] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:54 2022] PUT /identity/v3/projects/36ac94e0b9214eb0b0588be9ea5b4c29/users/f3b60e939fb24f6fbbb08c02bb984f9d/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:55.132256 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea355f-50ed-42d1-84ef-e8a2818c4d94 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:55.132596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea355f-50ed-42d1-84ef-e8a2818c4d94 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:55.132818 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea355f-50ed-42d1-84ef-e8a2818c4d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:55.145661 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-43ecc833-1f1e-4927-a8d4-5031a3ebbc2f None None] MFA Rules not processed for user `c0767eda97af49e2aac1e1f6c33ecc96`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:55.146910 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-43ecc833-1f1e-4927-a8d4-5031a3ebbc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:55.169800 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-09ea355f-50ed-42d1-84ef-e8a2818c4d94 None None] MFA Rules not processed for user `f3b60e939fb24f6fbbb08c02bb984f9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:55.171252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-09ea355f-50ed-42d1-84ef-e8a2818c4d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:55.245083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 939/1865] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:55.246043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecc833-1f1e-4927-a8d4-5031a3ebbc2f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:55.259843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 927/1866] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:59:55.260974 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea355f-50ed-42d1-84ef-e8a2818c4d94 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:55.262785 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-09ea355f-50ed-42d1-84ef-e8a2818c4d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:55.315615 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:55.335911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecc833-1f1e-4927-a8d4-5031a3ebbc2f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:55.336691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea355f-50ed-42d1-84ef-e8a2818c4d94 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:55.340719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b47b17-df69-4db6-9688-e92e7fb175c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:55.341392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b47b17-df69-4db6-9688-e92e7fb175c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:55.341550 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b854c0-1b19-4ac1-9313-e01faeb39890 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:55.341677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b47b17-df69-4db6-9688-e92e7fb175c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:55.341946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b47b17-df69-4db6-9688-e92e7fb175c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:55.342403 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b854c0-1b19-4ac1-9313-e01faeb39890 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:55.342743 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b854c0-1b19-4ac1-9313-e01faeb39890 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:55.343091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b854c0-1b19-4ac1-9313-e01faeb39890 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:55.345215 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a1b47b17-df69-4db6-9688-e92e7fb175c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:55.414212 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b854c0-1b19-4ac1-9313-e01faeb39890 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:55.417771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b854c0-1b19-4ac1-9313-e01faeb39890 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:55.426771 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b47b17-df69-4db6-9688-e92e7fb175c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:55.428503 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b47b17-df69-4db6-9688-e92e7fb175c7 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:55.429960 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 928/1867] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:55.430863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b854c0-1b19-4ac1-9313-e01faeb39890 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:55.445047 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 940/1868] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:55.516023 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:55.577082 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b854c0-1b19-4ac1-9313-e01faeb39890 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:55.588781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205cb51-10ec-47a4-9eed-1c94d1017e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:55.589582 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205cb51-10ec-47a4-9eed-1c94d1017e6a admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:55.589932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205cb51-10ec-47a4-9eed-1c94d1017e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:55.590282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205cb51-10ec-47a4-9eed-1c94d1017e6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:55.591075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205cb51-10ec-47a4-9eed-1c94d1017e6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:55.593086 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205cb51-10ec-47a4-9eed-1c94d1017e6a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:55.653224 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 929/1869] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 18:59:55 2022] POST /identity/v3/projects => generated 377 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:55.659110 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b47b17-df69-4db6-9688-e92e7fb175c7 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:55.684986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b47b17-df69-4db6-9688-e92e7fb175c7 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:55.880886 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00638bb-7b2a-4f01-9150-98e2983a674e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:55.881908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00638bb-7b2a-4f01-9150-98e2983a674e admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:55.882374 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00638bb-7b2a-4f01-9150-98e2983a674e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:55.882822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00638bb-7b2a-4f01-9150-98e2983a674e admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:55.883904 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00638bb-7b2a-4f01-9150-98e2983a674e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:55.887128 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00638bb-7b2a-4f01-9150-98e2983a674e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:55.951694 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 941/1870] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 18:59:55 2022] POST /identity/v3/users => generated 295 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:59:55.952858 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205cb51-10ec-47a4-9eed-1c94d1017e6a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:55.987746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205cb51-10ec-47a4-9eed-1c94d1017e6a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:56.013822 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c675c8-d69b-4e23-acda-759056b5045c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:56.016153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c675c8-d69b-4e23-acda-759056b5045c admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:56.016550 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c675c8-d69b-4e23-acda-759056b5045c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:56.017555 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c675c8-d69b-4e23-acda-759056b5045c admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:56.020504 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c675c8-d69b-4e23-acda-759056b5045c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:56.023872 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c675c8-d69b-4e23-acda-759056b5045c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:56.063428 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 930/1871] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:56.068124 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00638bb-7b2a-4f01-9150-98e2983a674e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:56.089109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00638bb-7b2a-4f01-9150-98e2983a674e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:56.127552 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eac38b-6e9b-4e4b-9506-ddb1d74db88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:56.128945 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eac38b-6e9b-4e4b-9506-ddb1d74db88b admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:56.129372 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eac38b-6e9b-4e4b-9506-ddb1d74db88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:56.129810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eac38b-6e9b-4e4b-9506-ddb1d74db88b admin admin] PATH_INFO: `/v3/projects/c698abe563e34ae4aa659b9d62d00be7/users/a6a496b5e78f42f1839abbc0c7e34db6/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:56.175485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eac38b-6e9b-4e4b-9506-ddb1d74db88b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c698abe563e34ae4aa659b9d62d00be7, user_id=a6a496b5e78f42f1839abbc0c7e34db6, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:56.178192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eac38b-6e9b-4e4b-9506-ddb1d74db88b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:56.217862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 942/1872] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:56 2022] PUT /identity/v3/projects/c698abe563e34ae4aa659b9d62d00be7/users/a6a496b5e78f42f1839abbc0c7e34db6/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:56.222250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c675c8-d69b-4e23-acda-759056b5045c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:56.245418 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c675c8-d69b-4e23-acda-759056b5045c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:56.383592 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d989908-6586-4ac1-b86b-920e04129855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:56.384293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d989908-6586-4ac1-b86b-920e04129855 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:56.384578 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d989908-6586-4ac1-b86b-920e04129855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:56.384930 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d989908-6586-4ac1-b86b-920e04129855 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:56.385587 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d989908-6586-4ac1-b86b-920e04129855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:56.387237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d989908-6586-4ac1-b86b-920e04129855 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:56.410740 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eac38b-6e9b-4e4b-9506-ddb1d74db88b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:56.411188 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 931/1873] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 18:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:59:56.445519 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eac38b-6e9b-4e4b-9506-ddb1d74db88b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:56.473491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fb4c6-5d4a-4d76-b6de-567d552c3908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:56.474683 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fb4c6-5d4a-4d76-b6de-567d552c3908 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:56.475199 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fb4c6-5d4a-4d76-b6de-567d552c3908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:56.475707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fb4c6-5d4a-4d76-b6de-567d552c3908 admin admin] PATH_INFO: `/v3/projects/c698abe563e34ae4aa659b9d62d00be7/users/a6a496b5e78f42f1839abbc0c7e34db6/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:56.481967 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06fb4c6-5d4a-4d76-b6de-567d552c3908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c698abe563e34ae4aa659b9d62d00be7, user_id=a6a496b5e78f42f1839abbc0c7e34db6, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:56.485507 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06fb4c6-5d4a-4d76-b6de-567d552c3908 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:56.527279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec89c2-a4a0-4c98-87f8-e511844e75a5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:56.527571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec89c2-a4a0-4c98-87f8-e511844e75a5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:56.527846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec89c2-a4a0-4c98-87f8-e511844e75a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:56.529181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 943/1874] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 18:59:56 2022] PUT /identity/v3/projects/c698abe563e34ae4aa659b9d62d00be7/users/a6a496b5e78f42f1839abbc0c7e34db6/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:56.584159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c7ec89c2-a4a0-4c98-87f8-e511844e75a5 None None] MFA Rules not processed for user `a6a496b5e78f42f1839abbc0c7e34db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:56.585314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c7ec89c2-a4a0-4c98-87f8-e511844e75a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:56.756195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fb4c6-5d4a-4d76-b6de-567d552c3908 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:56.769328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 932/1875] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:56.863285 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:56.881313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fb4c6-5d4a-4d76-b6de-567d552c3908 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:56.889357 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cca5c3-291d-4346-aef2-94f813aa326e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:56.890106 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cca5c3-291d-4346-aef2-94f813aa326e service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:56.890747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cca5c3-291d-4346-aef2-94f813aa326e service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:56.891317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cca5c3-291d-4346-aef2-94f813aa326e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:56.972660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cca5c3-291d-4346-aef2-94f813aa326e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:56.974982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cca5c3-291d-4346-aef2-94f813aa326e service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:57.000468 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 944/1876] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:59:58.366207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a93ca-484c-4f23-a6e5-3c3e962f6fb7 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:58.367331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a93ca-484c-4f23-a6e5-3c3e962f6fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:58.367760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a93ca-484c-4f23-a6e5-3c3e962f6fb7 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:58.369909 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 933/1877] 10.222.0.70 () {54 vars in 921 bytes} [Thu Jun 16 18:59:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:59:58.403706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76398aec-1efb-4c4c-85fe-747f2c070551 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:58.404791 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76398aec-1efb-4c4c-85fe-747f2c070551 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:58.405403 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76398aec-1efb-4c4c-85fe-747f2c070551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:58.450543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-76398aec-1efb-4c4c-85fe-747f2c070551 None None] MFA Rules not processed for user `8342da838fe4462b9de2817ad3ac4ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:58.452376 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-76398aec-1efb-4c4c-85fe-747f2c070551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:58.612146 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 945/1878] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Jun 16 18:59:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:58.630488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a93ca-484c-4f23-a6e5-3c3e962f6fb7 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:58.632514 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-680a93ca-484c-4f23-a6e5-3c3e962f6fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:58.662668 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a93ca-484c-4f23-a6e5-3c3e962f6fb7 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:58.672721 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39090b6-a423-4903-b6cd-03d882ab3403 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:58.673744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39090b6-a423-4903-b6cd-03d882ab3403 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:58.674205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39090b6-a423-4903-b6cd-03d882ab3403 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:58.674871 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39090b6-a423-4903-b6cd-03d882ab3403 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:58.692104 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39090b6-a423-4903-b6cd-03d882ab3403 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:58.695336 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39090b6-a423-4903-b6cd-03d882ab3403 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:58.714163 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 934/1879] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:59:58.785670 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76398aec-1efb-4c4c-85fe-747f2c070551 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:58.825996 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39090b6-a423-4903-b6cd-03d882ab3403 service nova] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:58.863561 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76398aec-1efb-4c4c-85fe-747f2c070551 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:58.868711 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:58.871594 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d223c7-395d-4c79-b9bb-a4606350862b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:58.873770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d223c7-395d-4c79-b9bb-a4606350862b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:58.874101 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d223c7-395d-4c79-b9bb-a4606350862b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:58.874414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d223c7-395d-4c79-b9bb-a4606350862b admin admin] PATH_INFO: `/v3/users/6a75fba2951a465ca3f064f654c8ae51` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:58.876899 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d223c7-395d-4c79-b9bb-a4606350862b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a75fba2951a465ca3f064f654c8ae51)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:58.879241 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d223c7-395d-4c79-b9bb-a4606350862b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:58.937061 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39090b6-a423-4903-b6cd-03d882ab3403 service nova] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:58.945721 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37239e9b-2c26-4c6d-a9f4-fae27ac6ca82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:58.946981 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37239e9b-2c26-4c6d-a9f4-fae27ac6ca82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:58.947440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37239e9b-2c26-4c6d-a9f4-fae27ac6ca82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:58.947885 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37239e9b-2c26-4c6d-a9f4-fae27ac6ca82 admin admin] PATH_INFO: `/v3/users/3a686db6e03b45f8b2813c33b35ec123` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:58.949331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37239e9b-2c26-4c6d-a9f4-fae27ac6ca82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a686db6e03b45f8b2813c33b35ec123)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:58.949650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:58.955433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37239e9b-2c26-4c6d-a9f4-fae27ac6ca82 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:58.968178 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b2d223c7-395d-4c79-b9bb-a4606350862b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a75fba2951a465ca3f064f654c8ae51'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:58.969552 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b2d223c7-395d-4c79-b9bb-a4606350862b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a75fba2951a465ca3f064f654c8ae51'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:58.970782 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b2d223c7-395d-4c79-b9bb-a4606350862b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a75fba2951a465ca3f064f654c8ae51'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:58.989709 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b2d223c7-395d-4c79-b9bb-a4606350862b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a75fba2951a465ca3f064f654c8ae51'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:59.004579 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b2d223c7-395d-4c79-b9bb-a4606350862b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a75fba2951a465ca3f064f654c8ae51'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:59.048284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 946/1880] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:58 2022] DELETE /identity/v3/users/6a75fba2951a465ca3f064f654c8ae51 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:59.054125 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05628c-791a-48c9-b190-7ca8b7209103 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:59.055108 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05628c-791a-48c9-b190-7ca8b7209103 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:59.056038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05628c-791a-48c9-b190-7ca8b7209103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:59.069613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-37239e9b-2c26-4c6d-a9f4-fae27ac6ca82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a686db6e03b45f8b2813c33b35ec123'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:59.070430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-37239e9b-2c26-4c6d-a9f4-fae27ac6ca82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a686db6e03b45f8b2813c33b35ec123'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:59.082397 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-37239e9b-2c26-4c6d-a9f4-fae27ac6ca82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a686db6e03b45f8b2813c33b35ec123'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:59.089134 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0b05628c-791a-48c9-b190-7ca8b7209103 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:59.090637 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0b05628c-791a-48c9-b190-7ca8b7209103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:59.103289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-37239e9b-2c26-4c6d-a9f4-fae27ac6ca82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a686db6e03b45f8b2813c33b35ec123'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:59.104641 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-37239e9b-2c26-4c6d-a9f4-fae27ac6ca82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a686db6e03b45f8b2813c33b35ec123'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:59.124145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 935/1881] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:58 2022] DELETE /identity/v3/users/3a686db6e03b45f8b2813c33b35ec123 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:59:59.129668 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca022af-ae90-405e-ac7d-5c22b4e1d3ec None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:59.129961 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca022af-ae90-405e-ac7d-5c22b4e1d3ec None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:59.130250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca022af-ae90-405e-ac7d-5c22b4e1d3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:59.152897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-fca022af-ae90-405e-ac7d-5c22b4e1d3ec None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:59.161396 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-fca022af-ae90-405e-ac7d-5c22b4e1d3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:59.181810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 947/1882] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:59:59.191810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05628c-791a-48c9-b190-7ca8b7209103 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:59.193324 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0b05628c-791a-48c9-b190-7ca8b7209103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:59.233632 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 936/1883] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:59:59.246176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca022af-ae90-405e-ac7d-5c22b4e1d3ec None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:59.248483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-fca022af-ae90-405e-ac7d-5c22b4e1d3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:59.262274 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05628c-791a-48c9-b190-7ca8b7209103 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:59.310307 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:59.326793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416e607-e724-4d96-b507-77a3f686431d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:59.328137 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416e607-e724-4d96-b507-77a3f686431d service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:59.328457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416e607-e724-4d96-b507-77a3f686431d service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:59.328759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416e607-e724-4d96-b507-77a3f686431d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:59.334832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca022af-ae90-405e-ac7d-5c22b4e1d3ec None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:59.339939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91820c78-8024-4f45-973b-05f8ad29c722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:59.340614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91820c78-8024-4f45-973b-05f8ad29c722 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:59.341400 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91820c78-8024-4f45-973b-05f8ad29c722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:59.341681 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91820c78-8024-4f45-973b-05f8ad29c722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:59.345354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416e607-e724-4d96-b507-77a3f686431d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:59.346864 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416e607-e724-4d96-b507-77a3f686431d service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:59.368755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91820c78-8024-4f45-973b-05f8ad29c722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:59.372587 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91820c78-8024-4f45-973b-05f8ad29c722 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:59.375825 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 948/1884] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:59:59.387918 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 937/1885] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:59:59.694856 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416e607-e724-4d96-b507-77a3f686431d service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:59.696524 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4416e607-e724-4d96-b507-77a3f686431d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:59.711501 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416e607-e724-4d96-b507-77a3f686431d service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:59.720214 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec55feb-f0f8-4bfb-ac15-2f93db7a4cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:59.721218 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec55feb-f0f8-4bfb-ac15-2f93db7a4cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:59.721514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec55feb-f0f8-4bfb-ac15-2f93db7a4cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:59.721764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec55feb-f0f8-4bfb-ac15-2f93db7a4cf3 admin admin] PATH_INFO: `/v3/users/b4ae53815d0242e598686d8ae742ad26` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:59.723443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec55feb-f0f8-4bfb-ac15-2f93db7a4cf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4ae53815d0242e598686d8ae742ad26)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:59.726295 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec55feb-f0f8-4bfb-ac15-2f93db7a4cf3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:59.736940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:59.844761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0ec55feb-f0f8-4bfb-ac15-2f93db7a4cf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4ae53815d0242e598686d8ae742ad26'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:59.845544 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0ec55feb-f0f8-4bfb-ac15-2f93db7a4cf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4ae53815d0242e598686d8ae742ad26'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:59.846164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0ec55feb-f0f8-4bfb-ac15-2f93db7a4cf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4ae53815d0242e598686d8ae742ad26'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:59.856951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0ec55feb-f0f8-4bfb-ac15-2f93db7a4cf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4ae53815d0242e598686d8ae742ad26'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:59.867559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0ec55feb-f0f8-4bfb-ac15-2f93db7a4cf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4ae53815d0242e598686d8ae742ad26'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:59.892803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 949/1886] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:59 2022] DELETE /identity/v3/users/b4ae53815d0242e598686d8ae742ad26 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:59.897485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17010db-4dec-4fa6-93bd-26d9b8346cc7 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:59.897833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17010db-4dec-4fa6-93bd-26d9b8346cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:59.898030 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17010db-4dec-4fa6-93bd-26d9b8346cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:59.920801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f17010db-4dec-4fa6-93bd-26d9b8346cc7 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:59.922668 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f17010db-4dec-4fa6-93bd-26d9b8346cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:59.932700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec55feb-f0f8-4bfb-ac15-2f93db7a4cf3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:59.934641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0ec55feb-f0f8-4bfb-ac15-2f93db7a4cf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:00.026219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 938/1887] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 18:59:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:00:00.027646 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17010db-4dec-4fa6-93bd-26d9b8346cc7 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:00.029668 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f17010db-4dec-4fa6-93bd-26d9b8346cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:00.048725 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17010db-4dec-4fa6-93bd-26d9b8346cc7 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:00.049268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec55feb-f0f8-4bfb-ac15-2f93db7a4cf3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:00.053792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef62242-7602-4c6f-ae86-01a47b27844d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:00.054661 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b639355-b50d-4097-949b-8af55c3d1e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:00.054803 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef62242-7602-4c6f-ae86-01a47b27844d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:00.055020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef62242-7602-4c6f-ae86-01a47b27844d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:00.055316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef62242-7602-4c6f-ae86-01a47b27844d admin admin] PATH_INFO: `/v3/projects/5059d65bb0674eb2bb799d3140e9c719` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:00.055906 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b639355-b50d-4097-949b-8af55c3d1e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:00.056183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b639355-b50d-4097-949b-8af55c3d1e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:00.056567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b639355-b50d-4097-949b-8af55c3d1e6d admin admin] PATH_INFO: `/v3/projects/de4b8df58fd64508a56495dfa6d5c63d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:00.056839 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef62242-7602-4c6f-ae86-01a47b27844d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5059d65bb0674eb2bb799d3140e9c719)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:00.057953 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b639355-b50d-4097-949b-8af55c3d1e6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=de4b8df58fd64508a56495dfa6d5c63d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:00.058624 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef62242-7602-4c6f-ae86-01a47b27844d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:00.060394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b639355-b50d-4097-949b-8af55c3d1e6d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:00.119075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0b639355-b50d-4097-949b-8af55c3d1e6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de4b8df58fd64508a56495dfa6d5c63d', 'request_id': 'req-0b639355-b50d-4097-949b-8af55c3d1e6d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.119397 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-bef62242-7602-4c6f-ae86-01a47b27844d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5059d65bb0674eb2bb799d3140e9c719', 'request_id': 'req-bef62242-7602-4c6f-ae86-01a47b27844d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.130063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0b639355-b50d-4097-949b-8af55c3d1e6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de4b8df58fd64508a56495dfa6d5c63d', 'request_id': 'req-0b639355-b50d-4097-949b-8af55c3d1e6d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.134047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-bef62242-7602-4c6f-ae86-01a47b27844d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5059d65bb0674eb2bb799d3140e9c719', 'request_id': 'req-bef62242-7602-4c6f-ae86-01a47b27844d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.137926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0b639355-b50d-4097-949b-8af55c3d1e6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de4b8df58fd64508a56495dfa6d5c63d', 'request_id': 'req-0b639355-b50d-4097-949b-8af55c3d1e6d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.146401 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-bef62242-7602-4c6f-ae86-01a47b27844d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5059d65bb0674eb2bb799d3140e9c719', 'request_id': 'req-bef62242-7602-4c6f-ae86-01a47b27844d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.152013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0b639355-b50d-4097-949b-8af55c3d1e6d admin admin] The token cache is being invalidate because project de4b8df58fd64508a56495dfa6d5c63d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:00.152319 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0b639355-b50d-4097-949b-8af55c3d1e6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.153115 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0b639355-b50d-4097-949b-8af55c3d1e6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.158686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 950/1888] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 18:59:59 2022] DELETE /identity/v3/projects/de4b8df58fd64508a56495dfa6d5c63d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:00:00.158686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b639355-b50d-4097-949b-8af55c3d1e6d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:00.161471 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0b639355-b50d-4097-949b-8af55c3d1e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:00.166922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-bef62242-7602-4c6f-ae86-01a47b27844d admin admin] The token cache is being invalidate because project 5059d65bb0674eb2bb799d3140e9c719 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:00.167402 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-bef62242-7602-4c6f-ae86-01a47b27844d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.169068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-bef62242-7602-4c6f-ae86-01a47b27844d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.172826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 939/1889] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:00 2022] DELETE /identity/v3/projects/5059d65bb0674eb2bb799d3140e9c719 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:00.179138 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef62242-7602-4c6f-ae86-01a47b27844d admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:00.181278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-bef62242-7602-4c6f-ae86-01a47b27844d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:00.190471 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:00.291721 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b639355-b50d-4097-949b-8af55c3d1e6d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:00.301757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8123e-3432-4a73-b3c7-3b3168e6da5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:00.302686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8123e-3432-4a73-b3c7-3b3168e6da5d service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:00.303158 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8123e-3432-4a73-b3c7-3b3168e6da5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:00.303500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8123e-3432-4a73-b3c7-3b3168e6da5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:00.304870 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef62242-7602-4c6f-ae86-01a47b27844d admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:00.305602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-79a8123e-3432-4a73-b3c7-3b3168e6da5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:00.318263 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:00.400469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b95ffe-b999-4a7b-aaa9-8ee9d962b33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:00.400469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b95ffe-b999-4a7b-aaa9-8ee9d962b33d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:00.401665 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b95ffe-b999-4a7b-aaa9-8ee9d962b33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:00.401665 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b95ffe-b999-4a7b-aaa9-8ee9d962b33d admin admin] PATH_INFO: `/v3/users/c0767eda97af49e2aac1e1f6c33ecc96` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:00.402655 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b95ffe-b999-4a7b-aaa9-8ee9d962b33d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0767eda97af49e2aac1e1f6c33ecc96)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:00.404635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b95ffe-b999-4a7b-aaa9-8ee9d962b33d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:00.408147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8123e-3432-4a73-b3c7-3b3168e6da5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:00.409770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8123e-3432-4a73-b3c7-3b3168e6da5d service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:00.421414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 951/1890] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:00:00.422973 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8123e-3432-4a73-b3c7-3b3168e6da5d service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:00.424811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-79a8123e-3432-4a73-b3c7-3b3168e6da5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:00.457303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8123e-3432-4a73-b3c7-3b3168e6da5d service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:00.464081 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71330aae-3472-4be0-8af9-f3ae13b89bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:00.470549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71330aae-3472-4be0-8af9-f3ae13b89bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:00.470895 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71330aae-3472-4be0-8af9-f3ae13b89bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:00.471166 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71330aae-3472-4be0-8af9-f3ae13b89bdd admin admin] PATH_INFO: `/v3/users/8485c6ad1ec7445db034a005ea18da02` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:00.472526 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c2b95ffe-b999-4a7b-aaa9-8ee9d962b33d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0767eda97af49e2aac1e1f6c33ecc96'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.472660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71330aae-3472-4be0-8af9-f3ae13b89bdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8485c6ad1ec7445db034a005ea18da02)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:00.473392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c2b95ffe-b999-4a7b-aaa9-8ee9d962b33d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0767eda97af49e2aac1e1f6c33ecc96'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.480396 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71330aae-3472-4be0-8af9-f3ae13b89bdd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:00.481448 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c2b95ffe-b999-4a7b-aaa9-8ee9d962b33d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0767eda97af49e2aac1e1f6c33ecc96'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.496456 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c2b95ffe-b999-4a7b-aaa9-8ee9d962b33d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0767eda97af49e2aac1e1f6c33ecc96'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.497215 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c2b95ffe-b999-4a7b-aaa9-8ee9d962b33d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0767eda97af49e2aac1e1f6c33ecc96'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.512549 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 940/1891] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:00:00 2022] DELETE /identity/v3/users/c0767eda97af49e2aac1e1f6c33ecc96 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:00.547682 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-71330aae-3472-4be0-8af9-f3ae13b89bdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8485c6ad1ec7445db034a005ea18da02'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.548401 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-71330aae-3472-4be0-8af9-f3ae13b89bdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8485c6ad1ec7445db034a005ea18da02'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.549018 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-71330aae-3472-4be0-8af9-f3ae13b89bdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8485c6ad1ec7445db034a005ea18da02'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.559610 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-71330aae-3472-4be0-8af9-f3ae13b89bdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8485c6ad1ec7445db034a005ea18da02'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.570998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-71330aae-3472-4be0-8af9-f3ae13b89bdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8485c6ad1ec7445db034a005ea18da02'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:00.605253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 952/1892] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:00:00 2022] DELETE /identity/v3/users/8485c6ad1ec7445db034a005ea18da02 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:01.099720 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b95ffe-b999-4a7b-aaa9-8ee9d962b33d admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:01.106163 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c2b95ffe-b999-4a7b-aaa9-8ee9d962b33d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:01.144443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71330aae-3472-4be0-8af9-f3ae13b89bdd admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:01.147082 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-71330aae-3472-4be0-8af9-f3ae13b89bdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:01.211851 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:01.248949 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71330aae-3472-4be0-8af9-f3ae13b89bdd admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:01.256796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cb680-4c66-4a79-a1d0-e12e3096ecb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:01.257869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cb680-4c66-4a79-a1d0-e12e3096ecb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:01.258324 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cb680-4c66-4a79-a1d0-e12e3096ecb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:01.258847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cb680-4c66-4a79-a1d0-e12e3096ecb5 admin admin] PATH_INFO: `/v3/projects/696a9a0ab1964ae5bc5f28ae777b5255` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:01.260395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cb680-4c66-4a79-a1d0-e12e3096ecb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=696a9a0ab1964ae5bc5f28ae777b5255)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:01.263524 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cb680-4c66-4a79-a1d0-e12e3096ecb5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:01.264257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b95ffe-b999-4a7b-aaa9-8ee9d962b33d admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:01.271017 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507269e-58c2-4381-85b4-ce46c5ee6970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:01.271904 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507269e-58c2-4381-85b4-ce46c5ee6970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:01.272184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507269e-58c2-4381-85b4-ce46c5ee6970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:01.272465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507269e-58c2-4381-85b4-ce46c5ee6970 admin admin] PATH_INFO: `/v3/projects/2245c4a8de4e4d97bf64a27c97c87fac` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:01.273758 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507269e-58c2-4381-85b4-ce46c5ee6970 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2245c4a8de4e4d97bf64a27c97c87fac)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:01.276570 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507269e-58c2-4381-85b4-ce46c5ee6970 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:01.326259 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:01.351097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3507269e-58c2-4381-85b4-ce46c5ee6970 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2245c4a8de4e4d97bf64a27c97c87fac', 'request_id': 'req-3507269e-58c2-4381-85b4-ce46c5ee6970'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.361635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3507269e-58c2-4381-85b4-ce46c5ee6970 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2245c4a8de4e4d97bf64a27c97c87fac', 'request_id': 'req-3507269e-58c2-4381-85b4-ce46c5ee6970'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.368262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-521cb680-4c66-4a79-a1d0-e12e3096ecb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '696a9a0ab1964ae5bc5f28ae777b5255', 'request_id': 'req-521cb680-4c66-4a79-a1d0-e12e3096ecb5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.369757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3507269e-58c2-4381-85b4-ce46c5ee6970 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2245c4a8de4e4d97bf64a27c97c87fac', 'request_id': 'req-3507269e-58c2-4381-85b4-ce46c5ee6970'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.378252 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-521cb680-4c66-4a79-a1d0-e12e3096ecb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '696a9a0ab1964ae5bc5f28ae777b5255', 'request_id': 'req-521cb680-4c66-4a79-a1d0-e12e3096ecb5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.385942 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-521cb680-4c66-4a79-a1d0-e12e3096ecb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '696a9a0ab1964ae5bc5f28ae777b5255', 'request_id': 'req-521cb680-4c66-4a79-a1d0-e12e3096ecb5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.389349 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3507269e-58c2-4381-85b4-ce46c5ee6970 admin admin] The token cache is being invalidate because project 2245c4a8de4e4d97bf64a27c97c87fac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:01.390263 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3507269e-58c2-4381-85b4-ce46c5ee6970 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.392004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3507269e-58c2-4381-85b4-ce46c5ee6970 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.398075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 941/1893] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:01 2022] DELETE /identity/v3/projects/2245c4a8de4e4d97bf64a27c97c87fac => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:01.398075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507269e-58c2-4381-85b4-ce46c5ee6970 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:01.398856 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3507269e-58c2-4381-85b4-ce46c5ee6970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:01.400119 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-521cb680-4c66-4a79-a1d0-e12e3096ecb5 admin admin] The token cache is being invalidate because project 696a9a0ab1964ae5bc5f28ae777b5255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:01.400441 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-521cb680-4c66-4a79-a1d0-e12e3096ecb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.401360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-521cb680-4c66-4a79-a1d0-e12e3096ecb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.403751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 953/1894] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:01 2022] DELETE /identity/v3/projects/696a9a0ab1964ae5bc5f28ae777b5255 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:01.404504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cb680-4c66-4a79-a1d0-e12e3096ecb5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:01.405972 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-521cb680-4c66-4a79-a1d0-e12e3096ecb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:01.481414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cb680-4c66-4a79-a1d0-e12e3096ecb5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:01.487515 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b81644-96d5-45e0-b2d0-8a728609a5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:01.488499 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b81644-96d5-45e0-b2d0-8a728609a5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:01.488784 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b81644-96d5-45e0-b2d0-8a728609a5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:01.489061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b81644-96d5-45e0-b2d0-8a728609a5b4 admin admin] PATH_INFO: `/v3/users/30dea9def7a242ab8d2b6efc275f9da5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:01.490226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507269e-58c2-4381-85b4-ce46c5ee6970 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:01.490724 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b81644-96d5-45e0-b2d0-8a728609a5b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30dea9def7a242ab8d2b6efc275f9da5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:01.492466 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b81644-96d5-45e0-b2d0-8a728609a5b4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:01.496649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b534c7-1cf9-4958-b7f8-ac4f8f54616a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:01.497793 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b534c7-1cf9-4958-b7f8-ac4f8f54616a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:01.498399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b534c7-1cf9-4958-b7f8-ac4f8f54616a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:01.498886 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b534c7-1cf9-4958-b7f8-ac4f8f54616a admin admin] PATH_INFO: `/v3/projects/f0d5c17586014530a8c21865450ea71e` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:01.500732 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b534c7-1cf9-4958-b7f8-ac4f8f54616a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0d5c17586014530a8c21865450ea71e)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:01.503225 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b534c7-1cf9-4958-b7f8-ac4f8f54616a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:01.554618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e4b81644-96d5-45e0-b2d0-8a728609a5b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30dea9def7a242ab8d2b6efc275f9da5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.555561 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e4b81644-96d5-45e0-b2d0-8a728609a5b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30dea9def7a242ab8d2b6efc275f9da5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.557020 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e4b81644-96d5-45e0-b2d0-8a728609a5b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30dea9def7a242ab8d2b6efc275f9da5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.565860 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a5b534c7-1cf9-4958-b7f8-ac4f8f54616a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0d5c17586014530a8c21865450ea71e', 'request_id': 'req-a5b534c7-1cf9-4958-b7f8-ac4f8f54616a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.566621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e4b81644-96d5-45e0-b2d0-8a728609a5b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30dea9def7a242ab8d2b6efc275f9da5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.575616 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e4b81644-96d5-45e0-b2d0-8a728609a5b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30dea9def7a242ab8d2b6efc275f9da5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.580176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a5b534c7-1cf9-4958-b7f8-ac4f8f54616a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0d5c17586014530a8c21865450ea71e', 'request_id': 'req-a5b534c7-1cf9-4958-b7f8-ac4f8f54616a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.583741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:01.588386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a5b534c7-1cf9-4958-b7f8-ac4f8f54616a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0d5c17586014530a8c21865450ea71e', 'request_id': 'req-a5b534c7-1cf9-4958-b7f8-ac4f8f54616a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.603038 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a5b534c7-1cf9-4958-b7f8-ac4f8f54616a admin admin] The token cache is being invalidate because project f0d5c17586014530a8c21865450ea71e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:01.603413 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a5b534c7-1cf9-4958-b7f8-ac4f8f54616a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.604747 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a5b534c7-1cf9-4958-b7f8-ac4f8f54616a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:01.607324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 942/1896] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:01 2022] DELETE /identity/v3/projects/f0d5c17586014530a8c21865450ea71e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:01.609535 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd284a3-1fb9-44f2-b59e-c322dffe64c6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:01.610008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd284a3-1fb9-44f2-b59e-c322dffe64c6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:01.610683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd284a3-1fb9-44f2-b59e-c322dffe64c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:01.614663 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 954/1896] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:00:01 2022] DELETE /identity/v3/users/30dea9def7a242ab8d2b6efc275f9da5 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:01.616958 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c4ec2-d8cf-44d6-b920-2a10221949ca None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:01.618298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c4ec2-d8cf-44d6-b920-2a10221949ca None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:01.619198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c4ec2-d8cf-44d6-b920-2a10221949ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:01.631724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-9fd284a3-1fb9-44f2-b59e-c322dffe64c6 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:01.634508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9fd284a3-1fb9-44f2-b59e-c322dffe64c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:01.652288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-615c4ec2-d8cf-44d6-b920-2a10221949ca None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:01.653753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-615c4ec2-d8cf-44d6-b920-2a10221949ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:01.690419 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:01.712198 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 943/1897] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:01.720079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4df38d-68c3-4d7a-a3f9-9d0b4a5858b8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:01.720389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4df38d-68c3-4d7a-a3f9-9d0b4a5858b8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:01.720671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4df38d-68c3-4d7a-a3f9-9d0b4a5858b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:01.743926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 955/1898] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:01.744120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-3a4df38d-68c3-4d7a-a3f9-9d0b4a5858b8 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:01.744795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3a4df38d-68c3-4d7a-a3f9-9d0b4a5858b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:01.746790 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e84f5-15af-4594-88c1-3c8c8c555d36 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:01.747269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e84f5-15af-4594-88c1-3c8c8c555d36 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:01.748326 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e84f5-15af-4594-88c1-3c8c8c555d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:01.754441 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 944/1899] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:00:01.759173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4df38d-68c3-4d7a-a3f9-9d0b4a5858b8 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:01.769820 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4df38d-68c3-4d7a-a3f9-9d0b4a5858b8 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:01.773493 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3e5e84f5-15af-4594-88c1-3c8c8c555d36 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:01.775009 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3e5e84f5-15af-4594-88c1-3c8c8c555d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:01.778842 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddba555-a307-45a7-870c-2a6494ce8c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:01.778842 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddba555-a307-45a7-870c-2a6494ce8c0a admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:01.778842 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddba555-a307-45a7-870c-2a6494ce8c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:01.778842 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddba555-a307-45a7-870c-2a6494ce8c0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:01.779353 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddba555-a307-45a7-870c-2a6494ce8c0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:00:01.779353 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddba555-a307-45a7-870c-2a6494ce8c0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:01.780310 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddba555-a307-45a7-870c-2a6494ce8c0a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:01.798158 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 956/1900] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:01.799130 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e84f5-15af-4594-88c1-3c8c8c555d36 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:01.802374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 945/1901] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:00:01.803308 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddba555-a307-45a7-870c-2a6494ce8c0a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:01.810756 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e84f5-15af-4594-88c1-3c8c8c555d36 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:01.815943 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76752b72-0582-4c94-b9c0-44baf8ec6453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:01.816596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76752b72-0582-4c94-b9c0-44baf8ec6453 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:01.816816 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddba555-a307-45a7-870c-2a6494ce8c0a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:01.816999 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76752b72-0582-4c94-b9c0-44baf8ec6453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:01.817289 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76752b72-0582-4c94-b9c0-44baf8ec6453 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:01.817883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76752b72-0582-4c94-b9c0-44baf8ec6453 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:00:01.818222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76752b72-0582-4c94-b9c0-44baf8ec6453 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:01.819889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76752b72-0582-4c94-b9c0-44baf8ec6453 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:01.823275 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52f8de-12af-40e3-9eea-fc7393b3449f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:01.823982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52f8de-12af-40e3-9eea-fc7393b3449f admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:01.824268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52f8de-12af-40e3-9eea-fc7393b3449f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:01.824548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52f8de-12af-40e3-9eea-fc7393b3449f admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:01.825271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a52f8de-12af-40e3-9eea-fc7393b3449f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:01.827242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a52f8de-12af-40e3-9eea-fc7393b3449f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:01.841103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 957/1902] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:00:01.845721 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76752b72-0582-4c94-b9c0-44baf8ec6453 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:01.858148 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76752b72-0582-4c94-b9c0-44baf8ec6453 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:01.865978 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 946/1903] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:01 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:01.867217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52f8de-12af-40e3-9eea-fc7393b3449f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:01.868026 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9124b-53c7-4a8e-b9b1-c8a6d2a402e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:01.868690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9124b-53c7-4a8e-b9b1-c8a6d2a402e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:01.868983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9124b-53c7-4a8e-b9b1-c8a6d2a402e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:01.870427 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9124b-53c7-4a8e-b9b1-c8a6d2a402e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:01.871290 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e9124b-53c7-4a8e-b9b1-c8a6d2a402e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:01.873156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e9124b-53c7-4a8e-b9b1-c8a6d2a402e2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:01.913367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 958/1904] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:01 2022] POST /identity/v3/projects => generated 409 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:00:01.914423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9124b-53c7-4a8e-b9b1-c8a6d2a402e2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:01.956960 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52f8de-12af-40e3-9eea-fc7393b3449f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:01.997669 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9124b-53c7-4a8e-b9b1-c8a6d2a402e2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:02.005070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5195b-1bc6-4a48-8f75-73138dea73ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:02.006333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5195b-1bc6-4a48-8f75-73138dea73ac admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:02.006934 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5195b-1bc6-4a48-8f75-73138dea73ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:02.007224 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5195b-1bc6-4a48-8f75-73138dea73ac admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:02.008159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5195b-1bc6-4a48-8f75-73138dea73ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:02.011580 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5195b-1bc6-4a48-8f75-73138dea73ac admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:02.023678 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f8ac7-0e46-40ad-bdaa-7b1c9e5dd058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:02.024584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f8ac7-0e46-40ad-bdaa-7b1c9e5dd058 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:02.025020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f8ac7-0e46-40ad-bdaa-7b1c9e5dd058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:02.025477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f8ac7-0e46-40ad-bdaa-7b1c9e5dd058 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:02.026605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f8ac7-0e46-40ad-bdaa-7b1c9e5dd058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:02.029498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f8ac7-0e46-40ad-bdaa-7b1c9e5dd058 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:02.061634 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 947/1905] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:01 2022] POST /identity/v3/users => generated 284 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:02.063564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f8ac7-0e46-40ad-bdaa-7b1c9e5dd058 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:02.071745 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-aa2f8ac7-0e46-40ad-bdaa-7b1c9e5dd058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:02.080477 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 959/1906] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:01 2022] POST /identity/v3/users => generated 305 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:02.086773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f8ac7-0e46-40ad-bdaa-7b1c9e5dd058 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:02.093386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:02.094817 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:02.095207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:02.095805 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb admin admin] PATH_INFO: `/v3/projects/8f6280684244433cb081e48d86f3c02b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:02.097140 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f6280684244433cb081e48d86f3c02b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:02.099364 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:02.101137 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5195b-1bc6-4a48-8f75-73138dea73ac admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:02.132296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5195b-1bc6-4a48-8f75-73138dea73ac admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:02.138957 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e54283-fa82-4701-86f0-0ddb9100277a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:02.140070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e54283-fa82-4701-86f0-0ddb9100277a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:02.141060 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e54283-fa82-4701-86f0-0ddb9100277a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:02.141476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e54283-fa82-4701-86f0-0ddb9100277a admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:02.142317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e54283-fa82-4701-86f0-0ddb9100277a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:02.144574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e54283-fa82-4701-86f0-0ddb9100277a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:02.160800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 960/1907] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:02.160800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e54283-fa82-4701-86f0-0ddb9100277a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:02.165769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:02.186415 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e54283-fa82-4701-86f0-0ddb9100277a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:02.189188 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f6280684244433cb081e48d86f3c02b', 'request_id': 'req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:02.206219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f6280684244433cb081e48d86f3c02b', 'request_id': 'req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:02.225295 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f6280684244433cb081e48d86f3c02b', 'request_id': 'req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:02.243259 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:02.257533 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb admin admin] The token cache is being invalidate because project 8f6280684244433cb081e48d86f3c02b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:02.257764 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:02.259421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:02.265765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 948/1908] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:02 2022] DELETE /identity/v3/projects/8f6280684244433cb081e48d86f3c02b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:02.267799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:02.269951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:02.345112 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8711a-d8c3-4f69-96e0-c7344cc8b4bb admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:02.350015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbb388-9dbd-413c-add8-cfaf0b6b9b61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:02.350894 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbb388-9dbd-413c-add8-cfaf0b6b9b61 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:02.351182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbb388-9dbd-413c-add8-cfaf0b6b9b61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:02.351468 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbb388-9dbd-413c-add8-cfaf0b6b9b61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:02.353795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-17cbb388-9dbd-413c-add8-cfaf0b6b9b61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:02.376493 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4056fb-89f3-4336-b90b-4a9529185b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:02.377044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4056fb-89f3-4336-b90b-4a9529185b2e admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:02.377339 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4056fb-89f3-4336-b90b-4a9529185b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:02.377567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4056fb-89f3-4336-b90b-4a9529185b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:02.378296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4056fb-89f3-4336-b90b-4a9529185b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:02.379961 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4056fb-89f3-4336-b90b-4a9529185b2e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:02.404931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 961/1909] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:02.405930 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4056fb-89f3-4336-b90b-4a9529185b2e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:02.408055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-af4056fb-89f3-4336-b90b-4a9529185b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:02.423373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4056fb-89f3-4336-b90b-4a9529185b2e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:02.428693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df934cf-46be-407a-9b25-e9350cec4010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:02.430029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df934cf-46be-407a-9b25-e9350cec4010 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:02.430319 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df934cf-46be-407a-9b25-e9350cec4010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:02.430703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df934cf-46be-407a-9b25-e9350cec4010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:02.431761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df934cf-46be-407a-9b25-e9350cec4010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:02.431924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbb388-9dbd-413c-add8-cfaf0b6b9b61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:02.433404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbb388-9dbd-413c-add8-cfaf0b6b9b61 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:02.433787 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df934cf-46be-407a-9b25-e9350cec4010 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:02.451326 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 949/1910] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:00:02.453655 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8480f-4ff4-49f1-a304-89658fab0cf2 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:02.453655 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8480f-4ff4-49f1-a304-89658fab0cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:02.453869 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8480f-4ff4-49f1-a304-89658fab0cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:02.465457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 962/1911] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:02.472586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df934cf-46be-407a-9b25-e9350cec4010 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:02.475449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0df934cf-46be-407a-9b25-e9350cec4010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:02.479005 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5ff8480f-4ff4-49f1-a304-89658fab0cf2 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:02.480870 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5ff8480f-4ff4-49f1-a304-89658fab0cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:02.493037 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 950/1912] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:02.493981 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8480f-4ff4-49f1-a304-89658fab0cf2 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:02.507018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8480f-4ff4-49f1-a304-89658fab0cf2 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:02.512343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df934cf-46be-407a-9b25-e9350cec4010 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:02.512636 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9e7a3-10fd-45c3-9d2f-0104b2999ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:02.513663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9e7a3-10fd-45c3-9d2f-0104b2999ea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:02.514052 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9e7a3-10fd-45c3-9d2f-0104b2999ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:02.514425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9e7a3-10fd-45c3-9d2f-0104b2999ea8 admin admin] PATH_INFO: `/v3/projects/7fb98b4301b343f392b4a466ad5c7b40/users/8dd2c077b8d3430f805e5dd4bc196fcd/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:02.527084 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc07615-01d0-443e-b4fb-e582bb0e2ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:02.527775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc07615-01d0-443e-b4fb-e582bb0e2ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:02.528291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc07615-01d0-443e-b4fb-e582bb0e2ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:02.528409 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc07615-01d0-443e-b4fb-e582bb0e2ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:02.529633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc07615-01d0-443e-b4fb-e582bb0e2ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:02.534634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc07615-01d0-443e-b4fb-e582bb0e2ab9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:02.541167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9e7a3-10fd-45c3-9d2f-0104b2999ea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb98b4301b343f392b4a466ad5c7b40, user_id=8dd2c077b8d3430f805e5dd4bc196fcd, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:02.544917 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9e7a3-10fd-45c3-9d2f-0104b2999ea8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:02.570710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 963/1913] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:02.580447 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 951/1914] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:02 2022] PUT /identity/v3/projects/7fb98b4301b343f392b4a466ad5c7b40/users/8dd2c077b8d3430f805e5dd4bc196fcd/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:02.580653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b687b0-083f-4ffc-a010-1495c3e4f4a3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:02.580653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b687b0-083f-4ffc-a010-1495c3e4f4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:02.580815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9e7a3-10fd-45c3-9d2f-0104b2999ea8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:02.580896 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b687b0-083f-4ffc-a010-1495c3e4f4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:02.621068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-18b687b0-083f-4ffc-a010-1495c3e4f4a3 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:02.622467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-18b687b0-083f-4ffc-a010-1495c3e4f4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:02.688931 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:02.750541 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9e7a3-10fd-45c3-9d2f-0104b2999ea8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:02.775047 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 964/1915] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:02.777811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8bc31-9474-4202-bc74-69e8d7bcd8b9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:02.779065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8bc31-9474-4202-bc74-69e8d7bcd8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:02.779372 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8bc31-9474-4202-bc74-69e8d7bcd8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:02.785323 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77345d00-d328-4c5a-9f4d-3de086d0f3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:02.786708 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77345d00-d328-4c5a-9f4d-3de086d0f3c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:02.791081 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77345d00-d328-4c5a-9f4d-3de086d0f3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:02.791691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77345d00-d328-4c5a-9f4d-3de086d0f3c9 admin admin] PATH_INFO: `/v3/projects/8805322d35e14b179b75a524f11f99ad/users/6061b415216a440c8c1240b62f7e77a3/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:02.826237 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6bd8bc31-9474-4202-bc74-69e8d7bcd8b9 None None] MFA Rules not processed for user `8dd2c077b8d3430f805e5dd4bc196fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:02.827938 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6bd8bc31-9474-4202-bc74-69e8d7bcd8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:02.902363 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77345d00-d328-4c5a-9f4d-3de086d0f3c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8805322d35e14b179b75a524f11f99ad, user_id=6061b415216a440c8c1240b62f7e77a3, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:02.909314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77345d00-d328-4c5a-9f4d-3de086d0f3c9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:02.962105 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 952/1916] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:02 2022] PUT /identity/v3/projects/8805322d35e14b179b75a524f11f99ad/users/6061b415216a440c8c1240b62f7e77a3/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:02.963173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77345d00-d328-4c5a-9f4d-3de086d0f3c9 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:03.015847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 965/1917] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:03.019680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eba70c-ce7e-4b25-af07-3f708ae600c3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:03.020610 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eba70c-ce7e-4b25-af07-3f708ae600c3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:03.021418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eba70c-ce7e-4b25-af07-3f708ae600c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:03.063669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77345d00-d328-4c5a-9f4d-3de086d0f3c9 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:03.073435 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece5fe8-11d4-4ca0-a338-87115d942374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:03.075074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece5fe8-11d4-4ca0-a338-87115d942374 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:03.075684 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece5fe8-11d4-4ca0-a338-87115d942374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:03.076099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece5fe8-11d4-4ca0-a338-87115d942374 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:03.078283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece5fe8-11d4-4ca0-a338-87115d942374 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:00:03.078742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece5fe8-11d4-4ca0-a338-87115d942374 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:03.081672 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece5fe8-11d4-4ca0-a338-87115d942374 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:03.105402 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 953/1918] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:00:03.106112 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece5fe8-11d4-4ca0-a338-87115d942374 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:03.112201 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3ece5fe8-11d4-4ca0-a338-87115d942374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:03.125029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c6eba70c-ce7e-4b25-af07-3f708ae600c3 None None] MFA Rules not processed for user `6061b415216a440c8c1240b62f7e77a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:03.126821 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c6eba70c-ce7e-4b25-af07-3f708ae600c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:03.163345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:03.202257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece5fe8-11d4-4ca0-a338-87115d942374 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:03.208344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf3e7e-b040-41a3-bcae-002d70ac89a1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8563a25d16134d639a2a8849976fed9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:03.209517 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf3e7e-b040-41a3-bcae-002d70ac89a1 service glance] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:03.209900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf3e7e-b040-41a3-bcae-002d70ac89a1 service glance] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:03.210272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf3e7e-b040-41a3-bcae-002d70ac89a1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:03.253924 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 966/1919] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:03 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:03.255380 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eba70c-ce7e-4b25-af07-3f708ae600c3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:03.274143 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eba70c-ce7e-4b25-af07-3f708ae600c3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:03.286048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebfc0e-f403-4462-8da9-cccbdcfa5f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:03.287166 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebfc0e-f403-4462-8da9-cccbdcfa5f8f admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:03.287846 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebfc0e-f403-4462-8da9-cccbdcfa5f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:03.288149 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebfc0e-f403-4462-8da9-cccbdcfa5f8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:03.290225 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebfc0e-f403-4462-8da9-cccbdcfa5f8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:03.296160 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebfc0e-f403-4462-8da9-cccbdcfa5f8f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:03.304488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf3e7e-b040-41a3-bcae-002d70ac89a1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:03.306030 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf3e7e-b040-41a3-bcae-002d70ac89a1 service glance] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:03.322031 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 954/1920] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:03 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:00:03.323017 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf3e7e-b040-41a3-bcae-002d70ac89a1 service glance] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:03.346952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf3e7e-b040-41a3-bcae-002d70ac89a1 service glance] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:03.354990 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 967/1921] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:03 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:03.356975 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebfc0e-f403-4462-8da9-cccbdcfa5f8f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:03.362019 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204fdaf7-3b9d-47f1-b302-892ab388a1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:03.363876 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fdaf7-3b9d-47f1-b302-892ab388a1aa admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:03.364170 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fdaf7-3b9d-47f1-b302-892ab388a1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:03.364930 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fdaf7-3b9d-47f1-b302-892ab388a1aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:03.367411 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204fdaf7-3b9d-47f1-b302-892ab388a1aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:03.379327 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204fdaf7-3b9d-47f1-b302-892ab388a1aa admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:03.473419 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:03.497522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 955/1922] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:03 2022] POST /identity/v3/projects => generated 409 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:03.506711 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204fdaf7-3b9d-47f1-b302-892ab388a1aa admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:03.558449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebfc0e-f403-4462-8da9-cccbdcfa5f8f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:03.637890 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8348e0-a7f4-498d-a90c-9133f0f75646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:03.638603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8348e0-a7f4-498d-a90c-9133f0f75646 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:03.638899 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8348e0-a7f4-498d-a90c-9133f0f75646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:03.639179 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8348e0-a7f4-498d-a90c-9133f0f75646 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:03.640072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8348e0-a7f4-498d-a90c-9133f0f75646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:03.642759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8348e0-a7f4-498d-a90c-9133f0f75646 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:03.655317 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204fdaf7-3b9d-47f1-b302-892ab388a1aa admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:03.674502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 968/1923] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:03 2022] POST /identity/v3/users => generated 293 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:03.677267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a25f0-394a-4aec-a9fe-599461dab277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:03.678068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a25f0-394a-4aec-a9fe-599461dab277 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:03.678358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a25f0-394a-4aec-a9fe-599461dab277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:03.678895 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8348e0-a7f4-498d-a90c-9133f0f75646 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:03.679016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a25f0-394a-4aec-a9fe-599461dab277 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:03.680205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a25f0-394a-4aec-a9fe-599461dab277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:03.681562 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a25f0-394a-4aec-a9fe-599461dab277 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:03.693694 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8348e0-a7f4-498d-a90c-9133f0f75646 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:03.699695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e294e4-5a58-4fa6-9624-b6146e845b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:03.700460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e294e4-5a58-4fa6-9624-b6146e845b18 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:03.700798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e294e4-5a58-4fa6-9624-b6146e845b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:03.701083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e294e4-5a58-4fa6-9624-b6146e845b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:03.702411 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e294e4-5a58-4fa6-9624-b6146e845b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:03.705165 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e294e4-5a58-4fa6-9624-b6146e845b18 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:03.723325 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 969/1924] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:03.728583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 956/1925] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:03 2022] POST /identity/v3/users => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:00:03.730949 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e294e4-5a58-4fa6-9624-b6146e845b18 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:03.732571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a25f0-394a-4aec-a9fe-599461dab277 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:03.758033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a25f0-394a-4aec-a9fe-599461dab277 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:03.760876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e294e4-5a58-4fa6-9624-b6146e845b18 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:03.763566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585beae-2d03-43bf-b5db-e5656feccbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:03.764322 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585beae-2d03-43bf-b5db-e5656feccbca admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:03.764616 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585beae-2d03-43bf-b5db-e5656feccbca admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:03.764896 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585beae-2d03-43bf-b5db-e5656feccbca admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:03.767420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585beae-2d03-43bf-b5db-e5656feccbca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:03.769420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585beae-2d03-43bf-b5db-e5656feccbca admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:03.777283 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e4b1e-f2c2-4970-9908-70085b9f3d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:03.778406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e4b1e-f2c2-4970-9908-70085b9f3d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:03.780113 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e4b1e-f2c2-4970-9908-70085b9f3d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:03.780559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e4b1e-f2c2-4970-9908-70085b9f3d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:03.782048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e4b1e-f2c2-4970-9908-70085b9f3d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:03.791812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e4b1e-f2c2-4970-9908-70085b9f3d4d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:03.800266 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 957/1926] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:03.801258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585beae-2d03-43bf-b5db-e5656feccbca admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:03.823159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 970/1927] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:00:03.824239 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585beae-2d03-43bf-b5db-e5656feccbca admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:03.829089 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e4b1e-f2c2-4970-9908-70085b9f3d4d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:03.835014 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76536a72-0357-458d-a16c-9059ab54986f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:03.836496 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76536a72-0357-458d-a16c-9059ab54986f admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:03.836956 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76536a72-0357-458d-a16c-9059ab54986f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:03.837381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76536a72-0357-458d-a16c-9059ab54986f admin admin] PATH_INFO: `/v3/projects/0d7c86d959014575a4f3ff1015fd2b6c/users/a12959eb2cb84628aa68d536b8cdcc2b/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:03.864264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e4b1e-f2c2-4970-9908-70085b9f3d4d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:03.869120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76536a72-0357-458d-a16c-9059ab54986f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d7c86d959014575a4f3ff1015fd2b6c, user_id=a12959eb2cb84628aa68d536b8cdcc2b, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:03.873546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4cb77-a677-4f13-b4e8-87f706f7280a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:03.874158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76536a72-0357-458d-a16c-9059ab54986f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:03.875030 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4cb77-a677-4f13-b4e8-87f706f7280a admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:03.876092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4cb77-a677-4f13-b4e8-87f706f7280a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:03.876814 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4cb77-a677-4f13-b4e8-87f706f7280a admin admin] PATH_INFO: `/v3/projects/1a28c98c71804c51bf5f54643279f258/users/a173a426625843008e2902f62048c5a4/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:03.884165 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:03.896875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 958/1928] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:03 2022] PUT /identity/v3/projects/0d7c86d959014575a4f3ff1015fd2b6c/users/a12959eb2cb84628aa68d536b8cdcc2b/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:03.900659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76536a72-0357-458d-a16c-9059ab54986f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:03.909304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4cb77-a677-4f13-b4e8-87f706f7280a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a28c98c71804c51bf5f54643279f258, user_id=a173a426625843008e2902f62048c5a4, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:03.914193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4cb77-a677-4f13-b4e8-87f706f7280a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:03.952542 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 971/1929] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:03 2022] PUT /identity/v3/projects/1a28c98c71804c51bf5f54643279f258/users/a173a426625843008e2902f62048c5a4/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:03.963811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b49a4e-399e-49dd-a2f2-c2d7acd1f70b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:03.964134 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b49a4e-399e-49dd-a2f2-c2d7acd1f70b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:03.964418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b49a4e-399e-49dd-a2f2-c2d7acd1f70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:04.021083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-26b49a4e-399e-49dd-a2f2-c2d7acd1f70b None None] MFA Rules not processed for user `a173a426625843008e2902f62048c5a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:04.023025 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-26b49a4e-399e-49dd-a2f2-c2d7acd1f70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:04.057101 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76536a72-0357-458d-a16c-9059ab54986f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:04.131478 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 972/1930] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:04.144251 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c8770-6246-4258-a481-d54e221d8402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:04.144937 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c8770-6246-4258-a481-d54e221d8402 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:04.145265 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c8770-6246-4258-a481-d54e221d8402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:04.145550 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c8770-6246-4258-a481-d54e221d8402 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:04.146071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b49a4e-399e-49dd-a2f2-c2d7acd1f70b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:04.146278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c8770-6246-4258-a481-d54e221d8402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:04.148786 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c8770-6246-4258-a481-d54e221d8402 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:04.149712 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-26b49a4e-399e-49dd-a2f2-c2d7acd1f70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:04.150727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:04.177571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 959/1931] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:04.181485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c8770-6246-4258-a481-d54e221d8402 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:04.200722 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c8770-6246-4258-a481-d54e221d8402 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:04.206948 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166f7b1-5ede-4832-89ca-49729ebdbaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:04.208012 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166f7b1-5ede-4832-89ca-49729ebdbaee admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:04.208409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166f7b1-5ede-4832-89ca-49729ebdbaee admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:04.208805 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166f7b1-5ede-4832-89ca-49729ebdbaee admin admin] PATH_INFO: `/v3/projects/0d7c86d959014575a4f3ff1015fd2b6c/users/a12959eb2cb84628aa68d536b8cdcc2b/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:04.211819 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166f7b1-5ede-4832-89ca-49729ebdbaee admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d7c86d959014575a4f3ff1015fd2b6c, user_id=a12959eb2cb84628aa68d536b8cdcc2b, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:04.214970 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166f7b1-5ede-4832-89ca-49729ebdbaee admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:04.237619 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 960/1932] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:04 2022] PUT /identity/v3/projects/0d7c86d959014575a4f3ff1015fd2b6c/users/a12959eb2cb84628aa68d536b8cdcc2b/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:04.250160 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41d9aa-64ad-483e-b471-9b3e405c9b9a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:04.250641 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41d9aa-64ad-483e-b471-9b3e405c9b9a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:04.250954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41d9aa-64ad-483e-b471-9b3e405c9b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:04.274515 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b49a4e-399e-49dd-a2f2-c2d7acd1f70b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:04.294859 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-0f41d9aa-64ad-483e-b471-9b3e405c9b9a None None] MFA Rules not processed for user `a12959eb2cb84628aa68d536b8cdcc2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:04.296259 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0f41d9aa-64ad-483e-b471-9b3e405c9b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:04.341248 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e631f-8e39-45d8-96a6-12c479dc40b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:04.342041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e631f-8e39-45d8-96a6-12c479dc40b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:04.342405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e631f-8e39-45d8-96a6-12c479dc40b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:04.342791 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e631f-8e39-45d8-96a6-12c479dc40b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:04.397200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 961/1933] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:04 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:04.405250 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:04.410076 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41d9aa-64ad-483e-b471-9b3e405c9b9a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:04.420182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41d9aa-64ad-483e-b471-9b3e405c9b9a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:04.421069 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e631f-8e39-45d8-96a6-12c479dc40b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:04.423329 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e631f-8e39-45d8-96a6-12c479dc40b2 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:04.424820 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ee888-102e-4a60-bfd0-f315f7cce765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:04.425516 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ee888-102e-4a60-bfd0-f315f7cce765 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:04.425802 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ee888-102e-4a60-bfd0-f315f7cce765 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:04.426082 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ee888-102e-4a60-bfd0-f315f7cce765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:04.432500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 973/1934] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:00:04.435923 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ee888-102e-4a60-bfd0-f315f7cce765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:04.437899 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ee888-102e-4a60-bfd0-f315f7cce765 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:04.449703 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 962/1935] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:04 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:00:04.639336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e631f-8e39-45d8-96a6-12c479dc40b2 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:04.650672 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e631f-8e39-45d8-96a6-12c479dc40b2 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:04.655068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52626f-bcab-4b62-a503-6b4c7314cd03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:04.655722 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52626f-bcab-4b62-a503-6b4c7314cd03 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:04.656594 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52626f-bcab-4b62-a503-6b4c7314cd03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:04.656886 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52626f-bcab-4b62-a503-6b4c7314cd03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:04.726204 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52626f-bcab-4b62-a503-6b4c7314cd03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:04.727706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52626f-bcab-4b62-a503-6b4c7314cd03 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:04.742003 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 974/1936] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:04 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:00:14.446760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=47,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:14.733059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=35 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:19.582752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ee888-102e-4a60-bfd0-f315f7cce765 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:19.695896 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ee888-102e-4a60-bfd0-f315f7cce765 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:19.701595 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e1e6b-831f-43ae-9136-542efb414cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:19.702628 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e1e6b-831f-43ae-9136-542efb414cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:19.703010 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e1e6b-831f-43ae-9136-542efb414cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:19.703293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e1e6b-831f-43ae-9136-542efb414cf4 admin admin] PATH_INFO: `/v3/users/6061b415216a440c8c1240b62f7e77a3` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:19.705437 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e1e6b-831f-43ae-9136-542efb414cf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6061b415216a440c8c1240b62f7e77a3)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:19.708584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e1e6b-831f-43ae-9136-542efb414cf4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:19.775181 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f25e1e6b-831f-43ae-9136-542efb414cf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6061b415216a440c8c1240b62f7e77a3'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:19.776262 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f25e1e6b-831f-43ae-9136-542efb414cf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6061b415216a440c8c1240b62f7e77a3'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:19.789019 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f25e1e6b-831f-43ae-9136-542efb414cf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6061b415216a440c8c1240b62f7e77a3'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:19.817459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f25e1e6b-831f-43ae-9136-542efb414cf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6061b415216a440c8c1240b62f7e77a3'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:19.818276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f25e1e6b-831f-43ae-9136-542efb414cf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6061b415216a440c8c1240b62f7e77a3'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:19.843359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 963/1937] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:00:19 2022] DELETE /identity/v3/users/6061b415216a440c8c1240b62f7e77a3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:19.848324 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609cdad-9a97-4145-8f61-dd2d9aef4970 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:19.849642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609cdad-9a97-4145-8f61-dd2d9aef4970 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:19.850200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609cdad-9a97-4145-8f61-dd2d9aef4970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:19.876084 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c609cdad-9a97-4145-8f61-dd2d9aef4970 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:19.877611 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c609cdad-9a97-4145-8f61-dd2d9aef4970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:19.948470 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 975/1938] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:19.966683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e1e6b-831f-43ae-9136-542efb414cf4 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:19.969292 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f25e1e6b-831f-43ae-9136-542efb414cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:20.058141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e1e6b-831f-43ae-9136-542efb414cf4 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:20.063482 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4006c-01f8-4cd2-9cc0-b4b3cdb69c75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:20.064187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4006c-01f8-4cd2-9cc0-b4b3cdb69c75 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:20.064505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4006c-01f8-4cd2-9cc0-b4b3cdb69c75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:20.064826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4006c-01f8-4cd2-9cc0-b4b3cdb69c75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:20.074788 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4006c-01f8-4cd2-9cc0-b4b3cdb69c75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:20.076214 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4006c-01f8-4cd2-9cc0-b4b3cdb69c75 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:20.088801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 964/1939] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:00:20.247620 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609cdad-9a97-4145-8f61-dd2d9aef4970 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:20.249666 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c609cdad-9a97-4145-8f61-dd2d9aef4970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:20.261868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609cdad-9a97-4145-8f61-dd2d9aef4970 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:20.268484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7a0c9-b0d5-44ae-b94d-a168ac7c09f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:20.269314 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7a0c9-b0d5-44ae-b94d-a168ac7c09f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:20.269598 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7a0c9-b0d5-44ae-b94d-a168ac7c09f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:20.269878 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7a0c9-b0d5-44ae-b94d-a168ac7c09f3 admin admin] PATH_INFO: `/v3/users/a173a426625843008e2902f62048c5a4` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:20.271122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7a0c9-b0d5-44ae-b94d-a168ac7c09f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a173a426625843008e2902f62048c5a4)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:20.273980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7a0c9-b0d5-44ae-b94d-a168ac7c09f3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:20.343598 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-48d7a0c9-b0d5-44ae-b94d-a168ac7c09f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a173a426625843008e2902f62048c5a4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:20.344318 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-48d7a0c9-b0d5-44ae-b94d-a168ac7c09f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a173a426625843008e2902f62048c5a4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:20.344888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-48d7a0c9-b0d5-44ae-b94d-a168ac7c09f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a173a426625843008e2902f62048c5a4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:20.352974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-48d7a0c9-b0d5-44ae-b94d-a168ac7c09f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a173a426625843008e2902f62048c5a4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:20.360548 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-48d7a0c9-b0d5-44ae-b94d-a168ac7c09f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a173a426625843008e2902f62048c5a4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:20.394509 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 976/1940] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:00:20 2022] DELETE /identity/v3/users/a173a426625843008e2902f62048c5a4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:20.400163 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dfcb9-063a-4429-90f8-8b5d37593f36 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:20.400721 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dfcb9-063a-4429-90f8-8b5d37593f36 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:20.401248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dfcb9-063a-4429-90f8-8b5d37593f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:20.421119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d96dfcb9-063a-4429-90f8-8b5d37593f36 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:20.422530 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d96dfcb9-063a-4429-90f8-8b5d37593f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:20.441669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:20.480055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7a0c9-b0d5-44ae-b94d-a168ac7c09f3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:20.481662 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-48d7a0c9-b0d5-44ae-b94d-a168ac7c09f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:20.503641 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 965/1941] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:20.515217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dfcb9-063a-4429-90f8-8b5d37593f36 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:20.517290 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d96dfcb9-063a-4429-90f8-8b5d37593f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:20.585122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7a0c9-b0d5-44ae-b94d-a168ac7c09f3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:20.591426 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:20.592315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:20.592885 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:20.594119 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38 admin admin] PATH_INFO: `/v3/projects/8805322d35e14b179b75a524f11f99ad` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:20.595466 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dfcb9-063a-4429-90f8-8b5d37593f36 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:20.595905 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8805322d35e14b179b75a524f11f99ad)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:20.597494 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:20.599189 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7a3db-f568-4195-a9f8-b98522765a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:20.600005 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7a3db-f568-4195-a9f8-b98522765a9b service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:20.600314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7a3db-f568-4195-a9f8-b98522765a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:20.601216 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7a3db-f568-4195-a9f8-b98522765a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:20.611331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7a3db-f568-4195-a9f8-b98522765a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:20.612773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7a3db-f568-4195-a9f8-b98522765a9b service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:20.614489 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:20.626959 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 966/1942] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:00:20.652215 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8805322d35e14b179b75a524f11f99ad', 'request_id': 'req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:20.660928 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8805322d35e14b179b75a524f11f99ad', 'request_id': 'req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:20.667960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8805322d35e14b179b75a524f11f99ad', 'request_id': 'req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:20.682842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38 admin admin] The token cache is being invalidate because project 8805322d35e14b179b75a524f11f99ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:20.683419 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:20.684453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:20.686103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 977/1943] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:20 2022] DELETE /identity/v3/projects/8805322d35e14b179b75a524f11f99ad => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:20.691506 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7a3db-f568-4195-a9f8-b98522765a9b service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:20.704171 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7a3db-f568-4195-a9f8-b98522765a9b service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:20.763090 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb1a9d-f992-4c48-a003-35cd67c17963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:20.764120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb1a9d-f992-4c48-a003-35cd67c17963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:20.764411 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb1a9d-f992-4c48-a003-35cd67c17963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:20.765409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb1a9d-f992-4c48-a003-35cd67c17963 admin admin] PATH_INFO: `/v3/users/a12959eb2cb84628aa68d536b8cdcc2b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:20.767035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb1a9d-f992-4c48-a003-35cd67c17963 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a12959eb2cb84628aa68d536b8cdcc2b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:20.768966 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb1a9d-f992-4c48-a003-35cd67c17963 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:20.834375 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e3fb1a9d-f992-4c48-a003-35cd67c17963 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a12959eb2cb84628aa68d536b8cdcc2b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:20.835316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e3fb1a9d-f992-4c48-a003-35cd67c17963 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a12959eb2cb84628aa68d536b8cdcc2b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:20.846480 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e3fb1a9d-f992-4c48-a003-35cd67c17963 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a12959eb2cb84628aa68d536b8cdcc2b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:20.855622 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:20.861814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e3fb1a9d-f992-4c48-a003-35cd67c17963 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a12959eb2cb84628aa68d536b8cdcc2b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:20.862637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e3fb1a9d-f992-4c48-a003-35cd67c17963 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a12959eb2cb84628aa68d536b8cdcc2b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:20.879588 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 967/1944] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:00:20 2022] DELETE /identity/v3/users/a12959eb2cb84628aa68d536b8cdcc2b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:21.006665 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:21.008972 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:21.025770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8976e1-1e4c-49d1-9f77-6cbfd3ba1d38 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:21.099185 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017e674-1a35-46ef-9fe3-13b1b59f07b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:21.100394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017e674-1a35-46ef-9fe3-13b1b59f07b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.100840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017e674-1a35-46ef-9fe3-13b1b59f07b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.101151 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017e674-1a35-46ef-9fe3-13b1b59f07b0 admin admin] PATH_INFO: `/v3/projects/1a28c98c71804c51bf5f54643279f258` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.102690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017e674-1a35-46ef-9fe3-13b1b59f07b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a28c98c71804c51bf5f54643279f258)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:21.104782 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017e674-1a35-46ef-9fe3-13b1b59f07b0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:21.165832 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c017e674-1a35-46ef-9fe3-13b1b59f07b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a28c98c71804c51bf5f54643279f258', 'request_id': 'req-c017e674-1a35-46ef-9fe3-13b1b59f07b0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:21.177006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c017e674-1a35-46ef-9fe3-13b1b59f07b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a28c98c71804c51bf5f54643279f258', 'request_id': 'req-c017e674-1a35-46ef-9fe3-13b1b59f07b0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:21.187900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c017e674-1a35-46ef-9fe3-13b1b59f07b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a28c98c71804c51bf5f54643279f258', 'request_id': 'req-c017e674-1a35-46ef-9fe3-13b1b59f07b0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:21.203210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c017e674-1a35-46ef-9fe3-13b1b59f07b0 admin admin] The token cache is being invalidate because project 1a28c98c71804c51bf5f54643279f258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:21.203311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c017e674-1a35-46ef-9fe3-13b1b59f07b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:21.203923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c017e674-1a35-46ef-9fe3-13b1b59f07b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:21.205929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 978/1945] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:21 2022] DELETE /identity/v3/projects/1a28c98c71804c51bf5f54643279f258 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:21.211049 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a2aed-fe75-47cb-b87b-1ee63431a3dc None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.211346 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a2aed-fe75-47cb-b87b-1ee63431a3dc None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.211669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a2aed-fe75-47cb-b87b-1ee63431a3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.232383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-367a2aed-fe75-47cb-b87b-1ee63431a3dc None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:21.234508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-367a2aed-fe75-47cb-b87b-1ee63431a3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:21.302939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 968/1946] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:21.311115 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894a919-5cbb-4275-9e9f-e032b304ec1d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.312163 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894a919-5cbb-4275-9e9f-e032b304ec1d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.312986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894a919-5cbb-4275-9e9f-e032b304ec1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.338284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3894a919-5cbb-4275-9e9f-e032b304ec1d None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:21.339755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3894a919-5cbb-4275-9e9f-e032b304ec1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:21.349287 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 979/1947] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:21.354594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a2aed-fe75-47cb-b87b-1ee63431a3dc None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:21.361766 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894a919-5cbb-4275-9e9f-e032b304ec1d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:21.363422 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3894a919-5cbb-4275-9e9f-e032b304ec1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:21.365153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a2aed-fe75-47cb-b87b-1ee63431a3dc None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:21.371816 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed3674-0534-4a03-af32-1e354cf70ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:21.373068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed3674-0534-4a03-af32-1e354cf70ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.373068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed3674-0534-4a03-af32-1e354cf70ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.373068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed3674-0534-4a03-af32-1e354cf70ee5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.374397 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed3674-0534-4a03-af32-1e354cf70ee5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:00:21.374588 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed3674-0534-4a03-af32-1e354cf70ee5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:21.377192 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed3674-0534-4a03-af32-1e354cf70ee5 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:21.379053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894a919-5cbb-4275-9e9f-e032b304ec1d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:21.385406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295539b-4f07-43fe-b344-24f56728a49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:21.386149 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295539b-4f07-43fe-b344-24f56728a49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.386438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295539b-4f07-43fe-b344-24f56728a49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.387222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295539b-4f07-43fe-b344-24f56728a49d admin admin] PATH_INFO: `/v3/projects/0d7c86d959014575a4f3ff1015fd2b6c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.388502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a295539b-4f07-43fe-b344-24f56728a49d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d7c86d959014575a4f3ff1015fd2b6c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:21.391721 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a295539b-4f07-43fe-b344-24f56728a49d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:21.395221 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 969/1948] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:00:21.397228 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed3674-0534-4a03-af32-1e354cf70ee5 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:21.407550 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed3674-0534-4a03-af32-1e354cf70ee5 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:21.412789 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389d1c5-c255-41f9-b6f6-292354dea0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:21.413431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389d1c5-c255-41f9-b6f6-292354dea0aa admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.413709 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389d1c5-c255-41f9-b6f6-292354dea0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.413984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389d1c5-c255-41f9-b6f6-292354dea0aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.415854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389d1c5-c255-41f9-b6f6-292354dea0aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:21.416482 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:21.418688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389d1c5-c255-41f9-b6f6-292354dea0aa admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:21.444650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a295539b-4f07-43fe-b344-24f56728a49d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d7c86d959014575a4f3ff1015fd2b6c', 'request_id': 'req-a295539b-4f07-43fe-b344-24f56728a49d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:21.449680 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 970/1949] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:21 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:21.453787 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a295539b-4f07-43fe-b344-24f56728a49d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d7c86d959014575a4f3ff1015fd2b6c', 'request_id': 'req-a295539b-4f07-43fe-b344-24f56728a49d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:21.454897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389d1c5-c255-41f9-b6f6-292354dea0aa admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:21.460939 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a295539b-4f07-43fe-b344-24f56728a49d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d7c86d959014575a4f3ff1015fd2b6c', 'request_id': 'req-a295539b-4f07-43fe-b344-24f56728a49d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:21.472612 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a295539b-4f07-43fe-b344-24f56728a49d admin admin] The token cache is being invalidate because project 0d7c86d959014575a4f3ff1015fd2b6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:21.472904 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a295539b-4f07-43fe-b344-24f56728a49d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:21.473620 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a295539b-4f07-43fe-b344-24f56728a49d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:21.475401 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 980/1950] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:21 2022] DELETE /identity/v3/projects/0d7c86d959014575a4f3ff1015fd2b6c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:00:21.480617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d4a87-5ffd-4130-97c7-8bf4516032fb None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.480997 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d4a87-5ffd-4130-97c7-8bf4516032fb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.481360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d4a87-5ffd-4130-97c7-8bf4516032fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.501103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-377d4a87-5ffd-4130-97c7-8bf4516032fb None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:21.502328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-377d4a87-5ffd-4130-97c7-8bf4516032fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:21.524341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389d1c5-c255-41f9-b6f6-292354dea0aa admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:21.529459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137208e3-b064-42e3-a426-3c8b4fa4e20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:21.530779 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137208e3-b064-42e3-a426-3c8b4fa4e20b admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.531073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137208e3-b064-42e3-a426-3c8b4fa4e20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.531420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137208e3-b064-42e3-a426-3c8b4fa4e20b admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.532052 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137208e3-b064-42e3-a426-3c8b4fa4e20b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:21.534046 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137208e3-b064-42e3-a426-3c8b4fa4e20b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:21.554723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 971/1951] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:21 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:21.558985 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137208e3-b064-42e3-a426-3c8b4fa4e20b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:21.560610 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-137208e3-b064-42e3-a426-3c8b4fa4e20b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:21.565814 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 981/1952] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:00:21.573434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137208e3-b064-42e3-a426-3c8b4fa4e20b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:21.573827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba869e-be7d-4188-bcd4-3adbc9524d5b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.574122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba869e-be7d-4188-bcd4-3adbc9524d5b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.574462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba869e-be7d-4188-bcd4-3adbc9524d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.582146 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d6934-3e59-494c-a081-414fd14b337f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:21.583989 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d6934-3e59-494c-a081-414fd14b337f admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.583989 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d6934-3e59-494c-a081-414fd14b337f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.583989 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d6934-3e59-494c-a081-414fd14b337f admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.584620 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d6934-3e59-494c-a081-414fd14b337f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:21.586367 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d6934-3e59-494c-a081-414fd14b337f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:21.596478 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-92ba869e-be7d-4188-bcd4-3adbc9524d5b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:21.597675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-92ba869e-be7d-4188-bcd4-3adbc9524d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:21.601916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 972/1953] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:00:21.603521 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d6934-3e59-494c-a081-414fd14b337f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:21.607022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 982/1954] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:00:21.611662 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba869e-be7d-4188-bcd4-3adbc9524d5b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:21.625992 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba869e-be7d-4188-bcd4-3adbc9524d5b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:21.630351 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d6934-3e59-494c-a081-414fd14b337f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:21.631557 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc992e96-34dd-45af-b433-fee2195166eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:21.632241 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc992e96-34dd-45af-b433-fee2195166eb admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.632523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc992e96-34dd-45af-b433-fee2195166eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.632800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc992e96-34dd-45af-b433-fee2195166eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.633530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc992e96-34dd-45af-b433-fee2195166eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:00:21.633834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc992e96-34dd-45af-b433-fee2195166eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:21.635476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc992e96-34dd-45af-b433-fee2195166eb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:21.636435 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb474a5b-16a3-4270-ab02-5f5889d6e28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:21.637528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb474a5b-16a3-4270-ab02-5f5889d6e28d admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.637818 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb474a5b-16a3-4270-ab02-5f5889d6e28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.638716 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb474a5b-16a3-4270-ab02-5f5889d6e28d admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.639773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb474a5b-16a3-4270-ab02-5f5889d6e28d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:21.641166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb474a5b-16a3-4270-ab02-5f5889d6e28d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:21.651424 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 983/1955] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:00:21.660999 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 973/1956] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:21.661171 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc992e96-34dd-45af-b433-fee2195166eb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:21.662346 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb474a5b-16a3-4270-ab02-5f5889d6e28d admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:21.670291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc992e96-34dd-45af-b433-fee2195166eb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:21.676911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2a969-88a9-4532-9d60-07248b3d8a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:21.678176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2a969-88a9-4532-9d60-07248b3d8a29 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.678176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2a969-88a9-4532-9d60-07248b3d8a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.678176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2a969-88a9-4532-9d60-07248b3d8a29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.679034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2a969-88a9-4532-9d60-07248b3d8a29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:21.679218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb474a5b-16a3-4270-ab02-5f5889d6e28d admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:21.680958 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2a969-88a9-4532-9d60-07248b3d8a29 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:21.685850 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5e143-3cc4-456b-8e57-ffb2c78819bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:21.686881 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5e143-3cc4-456b-8e57-ffb2c78819bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.687169 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5e143-3cc4-456b-8e57-ffb2c78819bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.687449 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5e143-3cc4-456b-8e57-ffb2c78819bd admin admin] PATH_INFO: `/v3/projects/b02e6f893b4240b684cef79c12c344a6/users/7067de54c427417b8fc1c1f3746eb6e9/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.711730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5e143-3cc4-456b-8e57-ffb2c78819bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b02e6f893b4240b684cef79c12c344a6, user_id=7067de54c427417b8fc1c1f3746eb6e9, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:21.714459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5e143-3cc4-456b-8e57-ffb2c78819bd admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:21.718320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 984/1957] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:21 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:21.718510 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:21.719934 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2a969-88a9-4532-9d60-07248b3d8a29 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:21.735928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 974/1958] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:21 2022] PUT /identity/v3/projects/b02e6f893b4240b684cef79c12c344a6/users/7067de54c427417b8fc1c1f3746eb6e9/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:00:21.746161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111ab76-f95f-4f7d-bb9a-4fbd3136aaed None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.746486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111ab76-f95f-4f7d-bb9a-4fbd3136aaed None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.746801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111ab76-f95f-4f7d-bb9a-4fbd3136aaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.780467 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-3111ab76-f95f-4f7d-bb9a-4fbd3136aaed None None] MFA Rules not processed for user `7067de54c427417b8fc1c1f3746eb6e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:21.782313 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3111ab76-f95f-4f7d-bb9a-4fbd3136aaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:21.807313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2a969-88a9-4532-9d60-07248b3d8a29 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:21.821001 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:21.852247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 975/1959] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:21.865195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111ab76-f95f-4f7d-bb9a-4fbd3136aaed None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:21.866711 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3111ab76-f95f-4f7d-bb9a-4fbd3136aaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:21.874834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f22d8d-f3fc-451b-bd19-b111b144870a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:21.875423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f22d8d-f3fc-451b-bd19-b111b144870a admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.875710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f22d8d-f3fc-451b-bd19-b111b144870a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.875988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f22d8d-f3fc-451b-bd19-b111b144870a admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.876677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f22d8d-f3fc-451b-bd19-b111b144870a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:21.878652 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f22d8d-f3fc-451b-bd19-b111b144870a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:21.902887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 985/1960] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:21 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:21.904270 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f22d8d-f3fc-451b-bd19-b111b144870a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:21.916062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f22d8d-f3fc-451b-bd19-b111b144870a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:21.921403 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b894625-72ae-4298-9861-ca325880aa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:21.922070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b894625-72ae-4298-9861-ca325880aa92 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.922420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b894625-72ae-4298-9861-ca325880aa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.922730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b894625-72ae-4298-9861-ca325880aa92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.923451 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b894625-72ae-4298-9861-ca325880aa92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:21.925043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b894625-72ae-4298-9861-ca325880aa92 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:21.928192 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111ab76-f95f-4f7d-bb9a-4fbd3136aaed None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:21.932082 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87f797-befd-4e11-bbac-500e176c97e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:21.932758 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87f797-befd-4e11-bbac-500e176c97e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.933042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87f797-befd-4e11-bbac-500e176c97e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.933319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87f797-befd-4e11-bbac-500e176c97e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.935038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 986/1961] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:00:21.939708 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b894625-72ae-4298-9861-ca325880aa92 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:21.943543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87f797-befd-4e11-bbac-500e176c97e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:21.945227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87f797-befd-4e11-bbac-500e176c97e2 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:21.949675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b894625-72ae-4298-9861-ca325880aa92 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:21.955158 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb325ee-3ee8-45a1-b993-d48ddfcc9490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:21.955831 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb325ee-3ee8-45a1-b993-d48ddfcc9490 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:21.956463 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb325ee-3ee8-45a1-b993-d48ddfcc9490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:21.956566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 976/1962] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:00:21.956642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb325ee-3ee8-45a1-b993-d48ddfcc9490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:21.957452 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb325ee-3ee8-45a1-b993-d48ddfcc9490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:21.959320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb325ee-3ee8-45a1-b993-d48ddfcc9490 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:21.971273 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 987/1963] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:00:21.976688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87f797-befd-4e11-bbac-500e176c97e2 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:21.988019 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb325ee-3ee8-45a1-b993-d48ddfcc9490 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:21.995024 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87f797-befd-4e11-bbac-500e176c97e2 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:22.003118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb325ee-3ee8-45a1-b993-d48ddfcc9490 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:22.003328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637d7b4-6bc6-47ef-b20a-b76f8ac3c5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:22.004139 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637d7b4-6bc6-47ef-b20a-b76f8ac3c5df admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:22.004430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637d7b4-6bc6-47ef-b20a-b76f8ac3c5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:22.004796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637d7b4-6bc6-47ef-b20a-b76f8ac3c5df admin admin] PATH_INFO: `/v3/projects/16741d513de64934a63abf8e754b1d17/users/ab77c441b2004df0a6556b4c7c0efa95/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:22.009286 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e88f3-e686-48e7-b4e4-b9269e72bcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:22.010343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e88f3-e686-48e7-b4e4-b9269e72bcb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:22.010654 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e88f3-e686-48e7-b4e4-b9269e72bcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:22.010930 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e88f3-e686-48e7-b4e4-b9269e72bcb6 admin admin] PATH_INFO: `/v3/users/7067de54c427417b8fc1c1f3746eb6e9` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:22.012236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e88f3-e686-48e7-b4e4-b9269e72bcb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7067de54c427417b8fc1c1f3746eb6e9)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:22.014176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e88f3-e686-48e7-b4e4-b9269e72bcb6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:22.023945 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637d7b4-6bc6-47ef-b20a-b76f8ac3c5df admin admin] RBAC: Authorizing `identity:create_grant(project_id=16741d513de64934a63abf8e754b1d17, user_id=ab77c441b2004df0a6556b4c7c0efa95, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:22.027296 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637d7b4-6bc6-47ef-b20a-b76f8ac3c5df admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:22.051895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 977/1964] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:21 2022] PUT /identity/v3/projects/16741d513de64934a63abf8e754b1d17/users/ab77c441b2004df0a6556b4c7c0efa95/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:22.064612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01beedc-3d2e-4dab-a874-7db2282b015f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:22.064909 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01beedc-3d2e-4dab-a874-7db2282b015f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:22.065206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01beedc-3d2e-4dab-a874-7db2282b015f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:22.073567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ce8e88f3-e686-48e7-b4e4-b9269e72bcb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7067de54c427417b8fc1c1f3746eb6e9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:22.074308 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ce8e88f3-e686-48e7-b4e4-b9269e72bcb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7067de54c427417b8fc1c1f3746eb6e9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:22.075038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ce8e88f3-e686-48e7-b4e4-b9269e72bcb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7067de54c427417b8fc1c1f3746eb6e9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:22.090092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ce8e88f3-e686-48e7-b4e4-b9269e72bcb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7067de54c427417b8fc1c1f3746eb6e9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:22.106437 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ce8e88f3-e686-48e7-b4e4-b9269e72bcb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7067de54c427417b8fc1c1f3746eb6e9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:22.111162 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f01beedc-3d2e-4dab-a874-7db2282b015f None None] MFA Rules not processed for user `ab77c441b2004df0a6556b4c7c0efa95`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:22.112582 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f01beedc-3d2e-4dab-a874-7db2282b015f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:22.127828 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 988/1965] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:00:21 2022] DELETE /identity/v3/users/7067de54c427417b8fc1c1f3746eb6e9 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:22.132419 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc58dc7-01df-4317-b93a-c9115dd9ae53 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:22.132717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc58dc7-01df-4317-b93a-c9115dd9ae53 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:22.133010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc58dc7-01df-4317-b93a-c9115dd9ae53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:22.153627 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-cfc58dc7-01df-4317-b93a-c9115dd9ae53 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:22.155883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cfc58dc7-01df-4317-b93a-c9115dd9ae53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:22.178988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:22.200667 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 978/1966] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:22.212879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01beedc-3d2e-4dab-a874-7db2282b015f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:22.214606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f01beedc-3d2e-4dab-a874-7db2282b015f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:22.223004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:22.258389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 989/1967] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:00:22.269839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc58dc7-01df-4317-b93a-c9115dd9ae53 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:22.271302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cfc58dc7-01df-4317-b93a-c9115dd9ae53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:22.302791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01beedc-3d2e-4dab-a874-7db2282b015f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:22.337659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc58dc7-01df-4317-b93a-c9115dd9ae53 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:22.341659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221029eb-127a-4d6e-9a1d-80f66f1ca10f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:22.342310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221029eb-127a-4d6e-9a1d-80f66f1ca10f service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:22.343040 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221029eb-127a-4d6e-9a1d-80f66f1ca10f service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:22.343669 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221029eb-127a-4d6e-9a1d-80f66f1ca10f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:22.355845 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221029eb-127a-4d6e-9a1d-80f66f1ca10f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:22.357832 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221029eb-127a-4d6e-9a1d-80f66f1ca10f service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:22.363972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736779c-43ef-4d62-b567-a881b64c5d55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:22.364816 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736779c-43ef-4d62-b567-a881b64c5d55 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:22.365118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736779c-43ef-4d62-b567-a881b64c5d55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:22.365400 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736779c-43ef-4d62-b567-a881b64c5d55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:22.373253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 990/1968] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:00:22.446062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736779c-43ef-4d62-b567-a881b64c5d55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:22.447572 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736779c-43ef-4d62-b567-a881b64c5d55 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:22.464810 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 979/1969] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:00:22.682219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221029eb-127a-4d6e-9a1d-80f66f1ca10f service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:22.687827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-221029eb-127a-4d6e-9a1d-80f66f1ca10f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:22.717499 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221029eb-127a-4d6e-9a1d-80f66f1ca10f service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:22.723264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89cef0-3011-4dcd-a0f4-0416c40ee148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:22.724257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89cef0-3011-4dcd-a0f4-0416c40ee148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:22.724543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89cef0-3011-4dcd-a0f4-0416c40ee148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:22.724812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89cef0-3011-4dcd-a0f4-0416c40ee148 admin admin] PATH_INFO: `/v3/projects/b02e6f893b4240b684cef79c12c344a6` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:22.726129 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89cef0-3011-4dcd-a0f4-0416c40ee148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b02e6f893b4240b684cef79c12c344a6)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:22.728081 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89cef0-3011-4dcd-a0f4-0416c40ee148 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:22.818758 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cf89cef0-3011-4dcd-a0f4-0416c40ee148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b02e6f893b4240b684cef79c12c344a6', 'request_id': 'req-cf89cef0-3011-4dcd-a0f4-0416c40ee148'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:22.827635 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cf89cef0-3011-4dcd-a0f4-0416c40ee148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b02e6f893b4240b684cef79c12c344a6', 'request_id': 'req-cf89cef0-3011-4dcd-a0f4-0416c40ee148'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:22.834734 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cf89cef0-3011-4dcd-a0f4-0416c40ee148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b02e6f893b4240b684cef79c12c344a6', 'request_id': 'req-cf89cef0-3011-4dcd-a0f4-0416c40ee148'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:22.850831 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cf89cef0-3011-4dcd-a0f4-0416c40ee148 admin admin] The token cache is being invalidate because project b02e6f893b4240b684cef79c12c344a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:22.851401 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cf89cef0-3011-4dcd-a0f4-0416c40ee148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:22.852068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cf89cef0-3011-4dcd-a0f4-0416c40ee148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:22.853709 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 991/1970] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:22 2022] DELETE /identity/v3/projects/b02e6f893b4240b684cef79c12c344a6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:22.860627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbef5d-e0ab-476d-bd65-92d8f15cc275 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:22.861149 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbef5d-e0ab-476d-bd65-92d8f15cc275 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:22.861246 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbef5d-e0ab-476d-bd65-92d8f15cc275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:22.880007 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-fdbbef5d-e0ab-476d-bd65-92d8f15cc275 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:22.881327 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-fdbbef5d-e0ab-476d-bd65-92d8f15cc275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:22.953877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 980/1971] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:22.962934 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a43822-8cd1-485c-9889-a9c1ac59d1b2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:22.963222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a43822-8cd1-485c-9889-a9c1ac59d1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:22.963499 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a43822-8cd1-485c-9889-a9c1ac59d1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:22.984270 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-27a43822-8cd1-485c-9889-a9c1ac59d1b2 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:22.985490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-27a43822-8cd1-485c-9889-a9c1ac59d1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:22.993469 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 992/1972] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:00:22.997646 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbef5d-e0ab-476d-bd65-92d8f15cc275 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:23.002554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:23.009254 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbef5d-e0ab-476d-bd65-92d8f15cc275 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:23.013955 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85b3f2-abe7-4d28-936e-cfad30fef6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:23.014815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85b3f2-abe7-4d28-936e-cfad30fef6e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:23.015104 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85b3f2-abe7-4d28-936e-cfad30fef6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:23.015385 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85b3f2-abe7-4d28-936e-cfad30fef6e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:23.016109 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85b3f2-abe7-4d28-936e-cfad30fef6e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:00:23.016419 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85b3f2-abe7-4d28-936e-cfad30fef6e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:23.018022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85b3f2-abe7-4d28-936e-cfad30fef6e3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:23.030708 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 981/1973] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:00:23.035363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a43822-8cd1-485c-9889-a9c1ac59d1b2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:23.045297 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a43822-8cd1-485c-9889-a9c1ac59d1b2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:23.050909 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bc7d0-a5f9-4877-96b7-211dfc3de51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:23.051571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bc7d0-a5f9-4877-96b7-211dfc3de51a admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:23.051854 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bc7d0-a5f9-4877-96b7-211dfc3de51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:23.052130 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bc7d0-a5f9-4877-96b7-211dfc3de51a admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:23.052834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bc7d0-a5f9-4877-96b7-211dfc3de51a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:23.054800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bc7d0-a5f9-4877-96b7-211dfc3de51a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:23.082105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 993/1974] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:23 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:23.086133 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85b3f2-abe7-4d28-936e-cfad30fef6e3 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:23.097404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85b3f2-abe7-4d28-936e-cfad30fef6e3 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:23.156058 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62e87b-4d6c-495f-a70f-ef54104c4835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:23.156636 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62e87b-4d6c-495f-a70f-ef54104c4835 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:23.156915 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62e87b-4d6c-495f-a70f-ef54104c4835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:23.157192 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62e87b-4d6c-495f-a70f-ef54104c4835 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:23.157884 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62e87b-4d6c-495f-a70f-ef54104c4835 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:23.159835 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62e87b-4d6c-495f-a70f-ef54104c4835 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:23.180677 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 982/1975] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:23 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:23.184742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bc7d0-a5f9-4877-96b7-211dfc3de51a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:23.195580 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bc7d0-a5f9-4877-96b7-211dfc3de51a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:23.200180 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4162e9-eda3-426d-b0ff-8b4c95e969f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:23.200867 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4162e9-eda3-426d-b0ff-8b4c95e969f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:23.201147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4162e9-eda3-426d-b0ff-8b4c95e969f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:23.201418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4162e9-eda3-426d-b0ff-8b4c95e969f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:23.202695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4162e9-eda3-426d-b0ff-8b4c95e969f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:23.204379 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4162e9-eda3-426d-b0ff-8b4c95e969f4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:23.214661 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 994/1976] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:23.219375 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62e87b-4d6c-495f-a70f-ef54104c4835 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:23.229353 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62e87b-4d6c-495f-a70f-ef54104c4835 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:23.234528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b00bfb-9238-4a80-860b-cb8e6a0641ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:23.235248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b00bfb-9238-4a80-860b-cb8e6a0641ba admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:23.235533 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b00bfb-9238-4a80-860b-cb8e6a0641ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:23.235808 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b00bfb-9238-4a80-860b-cb8e6a0641ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:23.236520 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b00bfb-9238-4a80-860b-cb8e6a0641ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:23.238106 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b00bfb-9238-4a80-860b-cb8e6a0641ba admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:23.250860 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 983/1977] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:00:23.252223 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4162e9-eda3-426d-b0ff-8b4c95e969f4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:23.263780 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4162e9-eda3-426d-b0ff-8b4c95e969f4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:23.271278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2198d9-bab9-41aa-ae3c-96878606dcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:23.272206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2198d9-bab9-41aa-ae3c-96878606dcb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:23.272490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2198d9-bab9-41aa-ae3c-96878606dcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:23.272768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2198d9-bab9-41aa-ae3c-96878606dcb5 admin admin] PATH_INFO: `/v3/projects/629634dc3dc54893b68cb9c9dc08505d/users/27db09b804fa426a83f103cf5dd116a8/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:23.289877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2198d9-bab9-41aa-ae3c-96878606dcb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=629634dc3dc54893b68cb9c9dc08505d, user_id=27db09b804fa426a83f103cf5dd116a8, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:23.292572 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2198d9-bab9-41aa-ae3c-96878606dcb5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:23.307774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 995/1978] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:23 2022] PUT /identity/v3/projects/629634dc3dc54893b68cb9c9dc08505d/users/27db09b804fa426a83f103cf5dd116a8/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:00:23.313075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708734a1-44c8-4473-9b44-44d48734aabc None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:23.313391 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708734a1-44c8-4473-9b44-44d48734aabc None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:23.313675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708734a1-44c8-4473-9b44-44d48734aabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:23.332790 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-708734a1-44c8-4473-9b44-44d48734aabc None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:23.334088 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-708734a1-44c8-4473-9b44-44d48734aabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:23.406754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 984/1979] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:23.419424 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2198d9-bab9-41aa-ae3c-96878606dcb5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:23.422153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8c2198d9-bab9-41aa-ae3c-96878606dcb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:23.444010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:23.483610 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2198d9-bab9-41aa-ae3c-96878606dcb5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:23.487370 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f89e8-5c4c-44db-bc8a-738e01bd2476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:23.488016 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f89e8-5c4c-44db-bc8a-738e01bd2476 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:23.488301 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f89e8-5c4c-44db-bc8a-738e01bd2476 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:23.488577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f89e8-5c4c-44db-bc8a-738e01bd2476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:23.498695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f89e8-5c4c-44db-bc8a-738e01bd2476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:23.500960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f89e8-5c4c-44db-bc8a-738e01bd2476 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:23.510251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 996/1980] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:00:29.937701 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0e2c9-7dc4-470d-8931-5c3378689c49 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:29.938207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0e2c9-7dc4-470d-8931-5c3378689c49 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:29.938441 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0e2c9-7dc4-470d-8931-5c3378689c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:29.976387 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-59c0e2c9-7dc4-470d-8931-5c3378689c49 None None] MFA Rules not processed for user `27db09b804fa426a83f103cf5dd116a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:29.977613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-59c0e2c9-7dc4-470d-8931-5c3378689c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:30.068117 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:30.078237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 985/1981] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:30.079497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f89e8-5c4c-44db-bc8a-738e01bd2476 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:30.094049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f89e8-5c4c-44db-bc8a-738e01bd2476 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:30.099921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa9ed8-cefe-4b88-a951-04d4d08b571a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:30.100592 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa9ed8-cefe-4b88-a951-04d4d08b571a admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:30.100874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa9ed8-cefe-4b88-a951-04d4d08b571a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:30.101146 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa9ed8-cefe-4b88-a951-04d4d08b571a admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:30.101843 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa9ed8-cefe-4b88-a951-04d4d08b571a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:30.104102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa9ed8-cefe-4b88-a951-04d4d08b571a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:30.141181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 997/1982] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:30 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:30.142390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c0e2c9-7dc4-470d-8931-5c3378689c49 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:30.154078 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c0e2c9-7dc4-470d-8931-5c3378689c49 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:30.263833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79e5db-e1ac-4891-b0b4-73f748c806f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:30.265220 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79e5db-e1ac-4891-b0b4-73f748c806f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:30.266243 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79e5db-e1ac-4891-b0b4-73f748c806f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:30.266243 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79e5db-e1ac-4891-b0b4-73f748c806f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:30.266776 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79e5db-e1ac-4891-b0b4-73f748c806f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:30.268810 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79e5db-e1ac-4891-b0b4-73f748c806f8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:30.302531 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 986/1983] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:30 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:30.307510 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa9ed8-cefe-4b88-a951-04d4d08b571a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:30.326057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa9ed8-cefe-4b88-a951-04d4d08b571a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:30.331906 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4e313-4c13-4e88-8682-e1bbdeea6a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:30.332588 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4e313-4c13-4e88-8682-e1bbdeea6a25 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:30.333487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4e313-4c13-4e88-8682-e1bbdeea6a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:30.333487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4e313-4c13-4e88-8682-e1bbdeea6a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:30.333883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4e313-4c13-4e88-8682-e1bbdeea6a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:30.335800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4e313-4c13-4e88-8682-e1bbdeea6a25 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:30.366834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 998/1984] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:30.370956 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79e5db-e1ac-4891-b0b4-73f748c806f8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:30.387612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79e5db-e1ac-4891-b0b4-73f748c806f8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:30.393888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9d3a8-2c49-4819-87b6-d833f4fc7017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:30.395079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9d3a8-2c49-4819-87b6-d833f4fc7017 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:30.395504 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9d3a8-2c49-4819-87b6-d833f4fc7017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:30.396142 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9d3a8-2c49-4819-87b6-d833f4fc7017 admin admin] PATH_INFO: `/v3/projects/59a97968d23943bba34843c82aa0b44d/users/6528d33ac137459facf75fa72e70cf9c/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:30.425678 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e9d3a8-2c49-4819-87b6-d833f4fc7017 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a97968d23943bba34843c82aa0b44d, user_id=6528d33ac137459facf75fa72e70cf9c, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:30.430533 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e9d3a8-2c49-4819-87b6-d833f4fc7017 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:30.456016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 987/1985] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:30 2022] PUT /identity/v3/projects/59a97968d23943bba34843c82aa0b44d/users/6528d33ac137459facf75fa72e70cf9c/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:00:30.459838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4e313-4c13-4e88-8682-e1bbdeea6a25 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:30.476311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4e313-4c13-4e88-8682-e1bbdeea6a25 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:30.600404 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db8b70-1eb1-40b0-b1a9-64ba39903b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:30.602370 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db8b70-1eb1-40b0-b1a9-64ba39903b4e admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:30.602702 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db8b70-1eb1-40b0-b1a9-64ba39903b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:30.602992 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db8b70-1eb1-40b0-b1a9-64ba39903b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:30.603949 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db8b70-1eb1-40b0-b1a9-64ba39903b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:30.609480 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db8b70-1eb1-40b0-b1a9-64ba39903b4e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:30.648117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 999/1986] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:30.652529 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9d3a8-2c49-4819-87b6-d833f4fc7017 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:30.670279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9d3a8-2c49-4819-87b6-d833f4fc7017 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:30.683265 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade89b6-9c6c-4c4a-8408-e88b011b1951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:30.684372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade89b6-9c6c-4c4a-8408-e88b011b1951 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:30.685330 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade89b6-9c6c-4c4a-8408-e88b011b1951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:30.689079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade89b6-9c6c-4c4a-8408-e88b011b1951 admin admin] PATH_INFO: `/v3/projects/59a97968d23943bba34843c82aa0b44d/users/6528d33ac137459facf75fa72e70cf9c/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:30.691614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade89b6-9c6c-4c4a-8408-e88b011b1951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a97968d23943bba34843c82aa0b44d, user_id=6528d33ac137459facf75fa72e70cf9c, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:30.699641 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade89b6-9c6c-4c4a-8408-e88b011b1951 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:30.742491 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 988/1987] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:30 2022] PUT /identity/v3/projects/59a97968d23943bba34843c82aa0b44d/users/6528d33ac137459facf75fa72e70cf9c/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:35.408654 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4343b8-576a-4d1b-ab7a-c1dc974d540f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:35.409171 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4343b8-576a-4d1b-ab7a-c1dc974d540f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:35.409265 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4343b8-576a-4d1b-ab7a-c1dc974d540f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:35.444007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1b4343b8-576a-4d1b-ab7a-c1dc974d540f None None] MFA Rules not processed for user `6528d33ac137459facf75fa72e70cf9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:35.445153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1b4343b8-576a-4d1b-ab7a-c1dc974d540f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:35.544621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1000/1988] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:35 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:35.558311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade89b6-9c6c-4c4a-8408-e88b011b1951 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:35.559990 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-cade89b6-9c6c-4c4a-8408-e88b011b1951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:35.640117 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade89b6-9c6c-4c4a-8408-e88b011b1951 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:35.645883 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192b98a-30c8-4b44-9837-3a25ed92c52f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:35.646546 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192b98a-30c8-4b44-9837-3a25ed92c52f service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:35.646863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192b98a-30c8-4b44-9837-3a25ed92c52f service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:35.647144 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192b98a-30c8-4b44-9837-3a25ed92c52f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:35.700333 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:35.714902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4192b98a-30c8-4b44-9837-3a25ed92c52f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:35.716506 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4192b98a-30c8-4b44-9837-3a25ed92c52f service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:35.726651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 989/1989] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:00:37.557880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136fb29-070b-4a18-b7de-3fa1d3354aae None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:37.558442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136fb29-070b-4a18-b7de-3fa1d3354aae None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:37.558442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136fb29-070b-4a18-b7de-3fa1d3354aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:37.580170 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a136fb29-070b-4a18-b7de-3fa1d3354aae None None] MFA Rules not processed for user `27db09b804fa426a83f103cf5dd116a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:37.581393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a136fb29-070b-4a18-b7de-3fa1d3354aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:37.592014 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1001/1990] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:40.549859 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192b98a-30c8-4b44-9837-3a25ed92c52f service nova] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:40.551319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4192b98a-30c8-4b44-9837-3a25ed92c52f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:40.619310 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192b98a-30c8-4b44-9837-3a25ed92c52f service nova] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:40.627662 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a22a5-7949-4f16-9e28-04567212e2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:40.628566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a22a5-7949-4f16-9e28-04567212e2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:40.628844 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a22a5-7949-4f16-9e28-04567212e2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:40.629150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a22a5-7949-4f16-9e28-04567212e2cf admin admin] PATH_INFO: `/v3/users/f3b60e939fb24f6fbbb08c02bb984f9d` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:40.630435 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a22a5-7949-4f16-9e28-04567212e2cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b60e939fb24f6fbbb08c02bb984f9d)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:40.632422 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a22a5-7949-4f16-9e28-04567212e2cf admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:40.686636 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7f7a22a5-7949-4f16-9e28-04567212e2cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b60e939fb24f6fbbb08c02bb984f9d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:40.687516 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7f7a22a5-7949-4f16-9e28-04567212e2cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b60e939fb24f6fbbb08c02bb984f9d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:40.695265 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7f7a22a5-7949-4f16-9e28-04567212e2cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b60e939fb24f6fbbb08c02bb984f9d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:40.707812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7f7a22a5-7949-4f16-9e28-04567212e2cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b60e939fb24f6fbbb08c02bb984f9d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:40.708531 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7f7a22a5-7949-4f16-9e28-04567212e2cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b60e939fb24f6fbbb08c02bb984f9d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:40.720827 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 990/1991] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:00:40 2022] DELETE /identity/v3/users/f3b60e939fb24f6fbbb08c02bb984f9d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:40.725489 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a4cbe-6799-4e43-9411-d27a1d66bf36 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:40.725975 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a4cbe-6799-4e43-9411-d27a1d66bf36 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:40.725975 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a4cbe-6799-4e43-9411-d27a1d66bf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:40.732024 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:40.747935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-383a4cbe-6799-4e43-9411-d27a1d66bf36 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:40.749914 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-383a4cbe-6799-4e43-9411-d27a1d66bf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:40.812824 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1002/1992] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:40.823216 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a22a5-7949-4f16-9e28-04567212e2cf admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:40.824947 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7f7a22a5-7949-4f16-9e28-04567212e2cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:40.891912 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a22a5-7949-4f16-9e28-04567212e2cf admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:40.896226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baccf7c-f2eb-4820-a2ec-e04d329a9126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:40.896874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baccf7c-f2eb-4820-a2ec-e04d329a9126 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:40.897152 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baccf7c-f2eb-4820-a2ec-e04d329a9126 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:40.897453 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baccf7c-f2eb-4820-a2ec-e04d329a9126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:40.909867 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baccf7c-f2eb-4820-a2ec-e04d329a9126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:40.911703 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baccf7c-f2eb-4820-a2ec-e04d329a9126 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:40.925077 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 991/1993] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:00:41.222860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a4cbe-6799-4e43-9411-d27a1d66bf36 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:41.224400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-383a4cbe-6799-4e43-9411-d27a1d66bf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:41.237869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a4cbe-6799-4e43-9411-d27a1d66bf36 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:41.242975 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8356c-f647-4f91-a37b-6f62b73f4577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:41.243891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8356c-f647-4f91-a37b-6f62b73f4577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:41.244253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8356c-f647-4f91-a37b-6f62b73f4577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:41.244540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8356c-f647-4f91-a37b-6f62b73f4577 admin admin] PATH_INFO: `/v3/projects/36ac94e0b9214eb0b0588be9ea5b4c29` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:41.245816 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8356c-f647-4f91-a37b-6f62b73f4577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36ac94e0b9214eb0b0588be9ea5b4c29)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:41.252897 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8356c-f647-4f91-a37b-6f62b73f4577 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:41.302947 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a1a8356c-f647-4f91-a37b-6f62b73f4577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36ac94e0b9214eb0b0588be9ea5b4c29', 'request_id': 'req-a1a8356c-f647-4f91-a37b-6f62b73f4577'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:41.310640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a1a8356c-f647-4f91-a37b-6f62b73f4577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36ac94e0b9214eb0b0588be9ea5b4c29', 'request_id': 'req-a1a8356c-f647-4f91-a37b-6f62b73f4577'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:41.317166 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a1a8356c-f647-4f91-a37b-6f62b73f4577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36ac94e0b9214eb0b0588be9ea5b4c29', 'request_id': 'req-a1a8356c-f647-4f91-a37b-6f62b73f4577'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:41.331792 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a1a8356c-f647-4f91-a37b-6f62b73f4577 admin admin] The token cache is being invalidate because project 36ac94e0b9214eb0b0588be9ea5b4c29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:41.332014 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a1a8356c-f647-4f91-a37b-6f62b73f4577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:41.332997 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a1a8356c-f647-4f91-a37b-6f62b73f4577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:41.335712 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1003/1994] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:41 2022] DELETE /identity/v3/projects/36ac94e0b9214eb0b0588be9ea5b4c29 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:41.340311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baccf7c-f2eb-4820-a2ec-e04d329a9126 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:41.351500 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baccf7c-f2eb-4820-a2ec-e04d329a9126 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:41.356431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:41.421158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef9f74-7dae-4728-bfcb-76482c9649d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:41.422009 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef9f74-7dae-4728-bfcb-76482c9649d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:41.422293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef9f74-7dae-4728-bfcb-76482c9649d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:41.422594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef9f74-7dae-4728-bfcb-76482c9649d2 admin admin] PATH_INFO: `/v3/users/a6a496b5e78f42f1839abbc0c7e34db6` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:41.423867 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef9f74-7dae-4728-bfcb-76482c9649d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6a496b5e78f42f1839abbc0c7e34db6)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:41.425799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef9f74-7dae-4728-bfcb-76482c9649d2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:41.483462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1eef9f74-7dae-4728-bfcb-76482c9649d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6a496b5e78f42f1839abbc0c7e34db6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:41.484161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1eef9f74-7dae-4728-bfcb-76482c9649d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6a496b5e78f42f1839abbc0c7e34db6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:41.492352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1eef9f74-7dae-4728-bfcb-76482c9649d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6a496b5e78f42f1839abbc0c7e34db6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:41.505769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1eef9f74-7dae-4728-bfcb-76482c9649d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6a496b5e78f42f1839abbc0c7e34db6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:41.506814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1eef9f74-7dae-4728-bfcb-76482c9649d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6a496b5e78f42f1839abbc0c7e34db6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:41.521476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 992/1995] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:00:41 2022] DELETE /identity/v3/users/a6a496b5e78f42f1839abbc0c7e34db6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:41.831010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8356c-f647-4f91-a37b-6f62b73f4577 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:41.833156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a1a8356c-f647-4f91-a37b-6f62b73f4577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:41.847745 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8356c-f647-4f91-a37b-6f62b73f4577 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:41.913732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f82886-32a2-4c3c-a55a-5dbfa31185c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:41.914617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f82886-32a2-4c3c-a55a-5dbfa31185c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:41.914901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f82886-32a2-4c3c-a55a-5dbfa31185c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:41.915240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f82886-32a2-4c3c-a55a-5dbfa31185c7 admin admin] PATH_INFO: `/v3/projects/c698abe563e34ae4aa659b9d62d00be7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:41.916564 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f82886-32a2-4c3c-a55a-5dbfa31185c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c698abe563e34ae4aa659b9d62d00be7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:41.918515 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f82886-32a2-4c3c-a55a-5dbfa31185c7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:41.955575 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:41.981697 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b1f82886-32a2-4c3c-a55a-5dbfa31185c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c698abe563e34ae4aa659b9d62d00be7', 'request_id': 'req-b1f82886-32a2-4c3c-a55a-5dbfa31185c7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:41.992120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b1f82886-32a2-4c3c-a55a-5dbfa31185c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c698abe563e34ae4aa659b9d62d00be7', 'request_id': 'req-b1f82886-32a2-4c3c-a55a-5dbfa31185c7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:41.999490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b1f82886-32a2-4c3c-a55a-5dbfa31185c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c698abe563e34ae4aa659b9d62d00be7', 'request_id': 'req-b1f82886-32a2-4c3c-a55a-5dbfa31185c7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:42.010068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b1f82886-32a2-4c3c-a55a-5dbfa31185c7 admin admin] The token cache is being invalidate because project c698abe563e34ae4aa659b9d62d00be7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:42.010354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b1f82886-32a2-4c3c-a55a-5dbfa31185c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:42.011015 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b1f82886-32a2-4c3c-a55a-5dbfa31185c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:42.012708 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1004/1996] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:41 2022] DELETE /identity/v3/projects/c698abe563e34ae4aa659b9d62d00be7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:42.017987 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea766e5c-1556-4ffb-8e86-485637682a95 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:42.018497 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea766e5c-1556-4ffb-8e86-485637682a95 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:42.018637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea766e5c-1556-4ffb-8e86-485637682a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:42.036408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ea766e5c-1556-4ffb-8e86-485637682a95 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:42.037596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ea766e5c-1556-4ffb-8e86-485637682a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:42.097485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 993/1997] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:42.105367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c269e9-d718-4863-8d2a-ae3e923dbc53 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:42.105663 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c269e9-d718-4863-8d2a-ae3e923dbc53 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:42.105946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c269e9-d718-4863-8d2a-ae3e923dbc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:42.125547 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-71c269e9-d718-4863-8d2a-ae3e923dbc53 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:42.126825 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-71c269e9-d718-4863-8d2a-ae3e923dbc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:42.134932 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1005/1998] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:42.139219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea766e5c-1556-4ffb-8e86-485637682a95 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:42.149058 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea766e5c-1556-4ffb-8e86-485637682a95 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:42.154485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f929fb-c0c7-49fd-9cab-d6fc48f15a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:42.155173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f929fb-c0c7-49fd-9cab-d6fc48f15a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:42.155501 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f929fb-c0c7-49fd-9cab-d6fc48f15a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:42.155778 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f929fb-c0c7-49fd-9cab-d6fc48f15a67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:42.156517 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f929fb-c0c7-49fd-9cab-d6fc48f15a67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:00:42.156828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f929fb-c0c7-49fd-9cab-d6fc48f15a67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:42.158439 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f929fb-c0c7-49fd-9cab-d6fc48f15a67 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:42.172235 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 994/1999] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:00:42.177888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c269e9-d718-4863-8d2a-ae3e923dbc53 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:42.193512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c269e9-d718-4863-8d2a-ae3e923dbc53 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:42.210344 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210795c9-2e99-4d1d-95e9-3304c60e6686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:42.211065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210795c9-2e99-4d1d-95e9-3304c60e6686 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:42.211349 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210795c9-2e99-4d1d-95e9-3304c60e6686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:42.211627 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210795c9-2e99-4d1d-95e9-3304c60e6686 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:42.212364 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210795c9-2e99-4d1d-95e9-3304c60e6686 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:42.214288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210795c9-2e99-4d1d-95e9-3304c60e6686 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:42.247612 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1006/2000] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:42 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:42.249657 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f929fb-c0c7-49fd-9cab-d6fc48f15a67 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:42.259680 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f929fb-c0c7-49fd-9cab-d6fc48f15a67 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:42.271636 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:42.321644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532f51d-e8a6-49f3-9dc0-64b73fe91f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:42.322326 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532f51d-e8a6-49f3-9dc0-64b73fe91f2e admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:42.322638 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532f51d-e8a6-49f3-9dc0-64b73fe91f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:42.322918 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532f51d-e8a6-49f3-9dc0-64b73fe91f2e admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:42.323608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532f51d-e8a6-49f3-9dc0-64b73fe91f2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:42.325618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532f51d-e8a6-49f3-9dc0-64b73fe91f2e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:42.350067 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 995/2001] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:42 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:00:42.354208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210795c9-2e99-4d1d-95e9-3304c60e6686 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:42.366113 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210795c9-2e99-4d1d-95e9-3304c60e6686 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:42.370881 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81230e8-bc5f-4664-9e67-b2e932ee2ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:42.371561 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81230e8-bc5f-4664-9e67-b2e932ee2ded admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:42.371842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81230e8-bc5f-4664-9e67-b2e932ee2ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:42.372114 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81230e8-bc5f-4664-9e67-b2e932ee2ded admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:42.372830 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81230e8-bc5f-4664-9e67-b2e932ee2ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:42.374740 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81230e8-bc5f-4664-9e67-b2e932ee2ded admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:42.386006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1007/2002] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:42.390805 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532f51d-e8a6-49f3-9dc0-64b73fe91f2e admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:42.402461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532f51d-e8a6-49f3-9dc0-64b73fe91f2e admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:42.408548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee921976-26b8-4d51-9a7c-b0b5ef53f7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:42.409217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee921976-26b8-4d51-9a7c-b0b5ef53f7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:42.409503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee921976-26b8-4d51-9a7c-b0b5ef53f7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:42.409783 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee921976-26b8-4d51-9a7c-b0b5ef53f7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:42.410505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee921976-26b8-4d51-9a7c-b0b5ef53f7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:42.412168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee921976-26b8-4d51-9a7c-b0b5ef53f7b9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:42.425342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 996/2003] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:42.428892 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81230e8-bc5f-4664-9e67-b2e932ee2ded admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:42.439707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81230e8-bc5f-4664-9e67-b2e932ee2ded admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:42.444352 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52deba-a309-4686-8bc3-ce2aee5bcad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:42.445265 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52deba-a309-4686-8bc3-ce2aee5bcad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:42.445541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52deba-a309-4686-8bc3-ce2aee5bcad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:42.445814 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52deba-a309-4686-8bc3-ce2aee5bcad7 admin admin] PATH_INFO: `/v3/projects/0dd282d30d344818a93b38a7e8fee3fd/users/ea69564a5db94ef485e530e6e349c1de/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:42.460727 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52deba-a309-4686-8bc3-ce2aee5bcad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dd282d30d344818a93b38a7e8fee3fd, user_id=ea69564a5db94ef485e530e6e349c1de, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:42.463491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52deba-a309-4686-8bc3-ce2aee5bcad7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:42.478730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1008/2004] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:42 2022] PUT /identity/v3/projects/0dd282d30d344818a93b38a7e8fee3fd/users/ea69564a5db94ef485e530e6e349c1de/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:42.489349 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb0e59-7181-4d84-b85b-c773edadca56 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:42.489730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb0e59-7181-4d84-b85b-c773edadca56 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:42.490152 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb0e59-7181-4d84-b85b-c773edadca56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:42.517981 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f4eb0e59-7181-4d84-b85b-c773edadca56 None None] MFA Rules not processed for user `ea69564a5db94ef485e530e6e349c1de`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:42.519223 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f4eb0e59-7181-4d84-b85b-c773edadca56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:42.590251 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 997/2005] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:42.595284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52deba-a309-4686-8bc3-ce2aee5bcad7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:42.663102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52deba-a309-4686-8bc3-ce2aee5bcad7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:42.667756 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd4050-52d2-49ce-8f57-77c06ae58b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:42.668418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd4050-52d2-49ce-8f57-77c06ae58b1f admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:42.668704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd4050-52d2-49ce-8f57-77c06ae58b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:42.668982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd4050-52d2-49ce-8f57-77c06ae58b1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:42.669681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd4050-52d2-49ce-8f57-77c06ae58b1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:42.672563 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd4050-52d2-49ce-8f57-77c06ae58b1f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:42.699586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1009/2006] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:42 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:00:42.704313 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb0e59-7181-4d84-b85b-c773edadca56 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:42.714968 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb0e59-7181-4d84-b85b-c773edadca56 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:42.774336 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103160f1-fdab-4045-bbe7-c65aed8f8407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:42.775069 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103160f1-fdab-4045-bbe7-c65aed8f8407 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:42.775389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103160f1-fdab-4045-bbe7-c65aed8f8407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:42.775668 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103160f1-fdab-4045-bbe7-c65aed8f8407 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:42.776686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103160f1-fdab-4045-bbe7-c65aed8f8407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:42.779176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103160f1-fdab-4045-bbe7-c65aed8f8407 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:42.808539 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 998/2007] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:42 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:42.814309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd4050-52d2-49ce-8f57-77c06ae58b1f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:42.824804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd4050-52d2-49ce-8f57-77c06ae58b1f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:42.829516 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701d881-f7e1-456a-a673-4d49744b8192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:42.830240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701d881-f7e1-456a-a673-4d49744b8192 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:42.830518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701d881-f7e1-456a-a673-4d49744b8192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:42.830816 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701d881-f7e1-456a-a673-4d49744b8192 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:42.831558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701d881-f7e1-456a-a673-4d49744b8192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:42.833148 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701d881-f7e1-456a-a673-4d49744b8192 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:42.843834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1010/2008] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:42.848015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103160f1-fdab-4045-bbe7-c65aed8f8407 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:42.858505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103160f1-fdab-4045-bbe7-c65aed8f8407 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:42.863316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4acdda-9a3e-454d-8f56-51b281a0867e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:42.864202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4acdda-9a3e-454d-8f56-51b281a0867e admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:42.864490 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4acdda-9a3e-454d-8f56-51b281a0867e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:42.864772 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4acdda-9a3e-454d-8f56-51b281a0867e admin admin] PATH_INFO: `/v3/projects/3e09543a93834a7f823c809c8e2ccbd4/users/4a6369859b1f4cac9eeeec02d6d04a97/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:42.876939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:42.881620 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4acdda-9a3e-454d-8f56-51b281a0867e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e09543a93834a7f823c809c8e2ccbd4, user_id=4a6369859b1f4cac9eeeec02d6d04a97, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:42.884951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4acdda-9a3e-454d-8f56-51b281a0867e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:42.898342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 999/2009] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:42 2022] PUT /identity/v3/projects/3e09543a93834a7f823c809c8e2ccbd4/users/4a6369859b1f4cac9eeeec02d6d04a97/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:42.902769 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701d881-f7e1-456a-a673-4d49744b8192 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:42.914048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701d881-f7e1-456a-a673-4d49744b8192 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:42.941803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:42.985007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e88d53-59b3-4051-8be9-3bdb9956c8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:42.985007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e88d53-59b3-4051-8be9-3bdb9956c8be admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:42.985007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e88d53-59b3-4051-8be9-3bdb9956c8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:42.985735 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e88d53-59b3-4051-8be9-3bdb9956c8be admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:42.985840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e88d53-59b3-4051-8be9-3bdb9956c8be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:42.987505 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e88d53-59b3-4051-8be9-3bdb9956c8be admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:42.998711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1011/2010] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:43.002993 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4acdda-9a3e-454d-8f56-51b281a0867e admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:43.013738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4acdda-9a3e-454d-8f56-51b281a0867e admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:43.018823 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6aec5-db22-42b8-8768-98e86b889b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:43.019739 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6aec5-db22-42b8-8768-98e86b889b1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:43.020015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6aec5-db22-42b8-8768-98e86b889b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:43.020284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6aec5-db22-42b8-8768-98e86b889b1c admin admin] PATH_INFO: `/v3/projects/3e09543a93834a7f823c809c8e2ccbd4/users/4a6369859b1f4cac9eeeec02d6d04a97/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:43.022311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6aec5-db22-42b8-8768-98e86b889b1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e09543a93834a7f823c809c8e2ccbd4, user_id=4a6369859b1f4cac9eeeec02d6d04a97, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:43.024921 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6aec5-db22-42b8-8768-98e86b889b1c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:43.036882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1000/2011] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:43 2022] PUT /identity/v3/projects/3e09543a93834a7f823c809c8e2ccbd4/users/4a6369859b1f4cac9eeeec02d6d04a97/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:43.048725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95690fe-0129-4d7e-87b9-53a6eb4dde58 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:43.049231 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95690fe-0129-4d7e-87b9-53a6eb4dde58 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:43.049735 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95690fe-0129-4d7e-87b9-53a6eb4dde58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:43.076571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c95690fe-0129-4d7e-87b9-53a6eb4dde58 None None] MFA Rules not processed for user `4a6369859b1f4cac9eeeec02d6d04a97`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:43.077761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c95690fe-0129-4d7e-87b9-53a6eb4dde58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:43.152298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1012/2012] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:43.165209 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6aec5-db22-42b8-8768-98e86b889b1c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:43.166874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-cee6aec5-db22-42b8-8768-98e86b889b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:43.226343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6aec5-db22-42b8-8768-98e86b889b1c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:43.230178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e0867-54d7-43fb-8784-64eb542852da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:43.230863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e0867-54d7-43fb-8784-64eb542852da service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:43.231144 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e0867-54d7-43fb-8784-64eb542852da service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:43.231427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e0867-54d7-43fb-8784-64eb542852da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:43.243173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e0867-54d7-43fb-8784-64eb542852da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:43.244623 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e0867-54d7-43fb-8784-64eb542852da service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:43.255436 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1001/2013] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:00:43.605012 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95690fe-0129-4d7e-87b9-53a6eb4dde58 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:43.614855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95690fe-0129-4d7e-87b9-53a6eb4dde58 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:43.619390 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d68df-4cc1-4952-8998-48e35fcb2c08 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a6369859b1f4cac9eeeec02d6d04a97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e09543a93834a7f823c809c8e2ccbd4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:43.620044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d68df-4cc1-4952-8998-48e35fcb2c08 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:43.620347 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d68df-4cc1-4952-8998-48e35fcb2c08 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:43.620631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d68df-4cc1-4952-8998-48e35fcb2c08 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:43.621348 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d68df-4cc1-4952-8998-48e35fcb2c08 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:00:43.621660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d68df-4cc1-4952-8998-48e35fcb2c08 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:43.623484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d68df-4cc1-4952-8998-48e35fcb2c08 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:43.635282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1013/2014] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:00:43.640142 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e0867-54d7-43fb-8784-64eb542852da service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:43.649667 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e0867-54d7-43fb-8784-64eb542852da service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:43.655478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c3263-4836-4b17-afc1-baae5436ef6c tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a6369859b1f4cac9eeeec02d6d04a97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e09543a93834a7f823c809c8e2ccbd4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:43.656381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c3263-4836-4b17-afc1-baae5436ef6c tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:43.656791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c3263-4836-4b17-afc1-baae5436ef6c tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:43.657209 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c3263-4836-4b17-afc1-baae5436ef6c tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:43.658202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c3263-4836-4b17-afc1-baae5436ef6c tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:43.660162 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c3263-4836-4b17-afc1-baae5436ef6c tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:43.689220 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1002/2015] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:43 2022] POST /identity/v3/projects => generated 355 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:43.693723 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d68df-4cc1-4952-8998-48e35fcb2c08 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:43.706668 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d68df-4cc1-4952-8998-48e35fcb2c08 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:43.769606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42449d-816a-45c2-8ada-f8f6343403a2 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a6369859b1f4cac9eeeec02d6d04a97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e09543a93834a7f823c809c8e2ccbd4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:43.770286 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42449d-816a-45c2-8ada-f8f6343403a2 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:43.770599 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42449d-816a-45c2-8ada-f8f6343403a2 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:43.770908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42449d-816a-45c2-8ada-f8f6343403a2 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:43.771669 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42449d-816a-45c2-8ada-f8f6343403a2 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:00:43.771980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42449d-816a-45c2-8ada-f8f6343403a2 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:43.773668 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42449d-816a-45c2-8ada-f8f6343403a2 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:43.788330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1014/2016] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:00:43.830383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c3263-4836-4b17-afc1-baae5436ef6c tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:43.840969 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c3263-4836-4b17-afc1-baae5436ef6c tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:43.845695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a769234-5a2f-4483-a0df-b2db2d966a66 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a6369859b1f4cac9eeeec02d6d04a97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e09543a93834a7f823c809c8e2ccbd4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:43.846560 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a769234-5a2f-4483-a0df-b2db2d966a66 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:43.846886 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a769234-5a2f-4483-a0df-b2db2d966a66 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:43.847165 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a769234-5a2f-4483-a0df-b2db2d966a66 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] PATH_INFO: `/v3/projects/aa8adeb529bc4d52aeba9a86af1819fa` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:43.848353 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a769234-5a2f-4483-a0df-b2db2d966a66 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] RBAC: Authorizing `identity:delete_project(project_id=aa8adeb529bc4d52aeba9a86af1819fa)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:43.850256 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a769234-5a2f-4483-a0df-b2db2d966a66 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:43.921570 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a769234-5a2f-4483-a0df-b2db2d966a66 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa8adeb529bc4d52aeba9a86af1819fa', 'request_id': 'req-9a769234-5a2f-4483-a0df-b2db2d966a66'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:43.929498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a769234-5a2f-4483-a0df-b2db2d966a66 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa8adeb529bc4d52aeba9a86af1819fa', 'request_id': 'req-9a769234-5a2f-4483-a0df-b2db2d966a66'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:43.937499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a769234-5a2f-4483-a0df-b2db2d966a66 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa8adeb529bc4d52aeba9a86af1819fa', 'request_id': 'req-9a769234-5a2f-4483-a0df-b2db2d966a66'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:43.955037 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a769234-5a2f-4483-a0df-b2db2d966a66 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] The token cache is being invalidate because project aa8adeb529bc4d52aeba9a86af1819fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:43.955271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a769234-5a2f-4483-a0df-b2db2d966a66 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:43.956344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a769234-5a2f-4483-a0df-b2db2d966a66 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:43.961094 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1003/2017] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:43 2022] DELETE /identity/v3/projects/aa8adeb529bc4d52aeba9a86af1819fa => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:44.278227 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42449d-816a-45c2-8ada-f8f6343403a2 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:44.281266 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6d42449d-816a-45c2-8ada-f8f6343403a2 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:44.325370 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:44.384343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42449d-816a-45c2-8ada-f8f6343403a2 tempest-VolumeQuotaClassesTest-118756075 tempest-VolumeQuotaClassesTest-118756075-project-admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:44.503103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba03c1-8dcf-4207-89f0-ff762890aeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:44.504120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba03c1-8dcf-4207-89f0-ff762890aeef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:44.507951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba03c1-8dcf-4207-89f0-ff762890aeef admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:44.509773 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba03c1-8dcf-4207-89f0-ff762890aeef admin admin] PATH_INFO: `/v3/users/ea69564a5db94ef485e530e6e349c1de` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:44.515058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba03c1-8dcf-4207-89f0-ff762890aeef admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea69564a5db94ef485e530e6e349c1de)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:44.522162 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba03c1-8dcf-4207-89f0-ff762890aeef admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:44.612750 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e0ba03c1-8dcf-4207-89f0-ff762890aeef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea69564a5db94ef485e530e6e349c1de'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:44.615258 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e0ba03c1-8dcf-4207-89f0-ff762890aeef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea69564a5db94ef485e530e6e349c1de'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:44.615876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e0ba03c1-8dcf-4207-89f0-ff762890aeef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea69564a5db94ef485e530e6e349c1de'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:44.631568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e0ba03c1-8dcf-4207-89f0-ff762890aeef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea69564a5db94ef485e530e6e349c1de'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:44.654920 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e0ba03c1-8dcf-4207-89f0-ff762890aeef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea69564a5db94ef485e530e6e349c1de'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:44.692062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1015/2018] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:00:44 2022] DELETE /identity/v3/users/ea69564a5db94ef485e530e6e349c1de => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:44.696900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfc468-daf1-4f89-a279-2e1c434fe9ab None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:44.697455 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfc468-daf1-4f89-a279-2e1c434fe9ab None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:44.697531 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfc468-daf1-4f89-a279-2e1c434fe9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:44.721469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-47cfc468-daf1-4f89-a279-2e1c434fe9ab None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:44.722344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-47cfc468-daf1-4f89-a279-2e1c434fe9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:44.779196 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:44.825238 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1004/2019] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:44.837453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba03c1-8dcf-4207-89f0-ff762890aeef admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:44.839867 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e0ba03c1-8dcf-4207-89f0-ff762890aeef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:44.929430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba03c1-8dcf-4207-89f0-ff762890aeef admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:44.933759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396e2ba-e585-4e7b-8581-aaa1c923147e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:44.934629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396e2ba-e585-4e7b-8581-aaa1c923147e service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:44.935013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396e2ba-e585-4e7b-8581-aaa1c923147e service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:44.935441 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396e2ba-e585-4e7b-8581-aaa1c923147e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:44.947751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396e2ba-e585-4e7b-8581-aaa1c923147e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:44.949287 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396e2ba-e585-4e7b-8581-aaa1c923147e service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:44.955209 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:44.964510 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1016/2020] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:00:45.353586 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfc468-daf1-4f89-a279-2e1c434fe9ab None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:45.355302 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-47cfc468-daf1-4f89-a279-2e1c434fe9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:45.386599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfc468-daf1-4f89-a279-2e1c434fe9ab None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:45.393507 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956e6c1-519e-4604-a85b-263816f2e9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:45.394535 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956e6c1-519e-4604-a85b-263816f2e9ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:45.394864 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956e6c1-519e-4604-a85b-263816f2e9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:45.395143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956e6c1-519e-4604-a85b-263816f2e9ad admin admin] PATH_INFO: `/v3/projects/0dd282d30d344818a93b38a7e8fee3fd` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:45.396375 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956e6c1-519e-4604-a85b-263816f2e9ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dd282d30d344818a93b38a7e8fee3fd)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:45.398588 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956e6c1-519e-4604-a85b-263816f2e9ad admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:45.491717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c956e6c1-519e-4604-a85b-263816f2e9ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dd282d30d344818a93b38a7e8fee3fd', 'request_id': 'req-c956e6c1-519e-4604-a85b-263816f2e9ad'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:45.514988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c956e6c1-519e-4604-a85b-263816f2e9ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dd282d30d344818a93b38a7e8fee3fd', 'request_id': 'req-c956e6c1-519e-4604-a85b-263816f2e9ad'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:45.523592 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c956e6c1-519e-4604-a85b-263816f2e9ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dd282d30d344818a93b38a7e8fee3fd', 'request_id': 'req-c956e6c1-519e-4604-a85b-263816f2e9ad'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:45.552422 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c956e6c1-519e-4604-a85b-263816f2e9ad admin admin] The token cache is being invalidate because project 0dd282d30d344818a93b38a7e8fee3fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:45.552737 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c956e6c1-519e-4604-a85b-263816f2e9ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:45.553445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c956e6c1-519e-4604-a85b-263816f2e9ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:45.557261 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1005/2021] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:45 2022] DELETE /identity/v3/projects/0dd282d30d344818a93b38a7e8fee3fd => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:45.561614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396e2ba-e585-4e7b-8581-aaa1c923147e service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:45.578621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396e2ba-e585-4e7b-8581-aaa1c923147e service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:45.727152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b694b31-b39b-406c-bc9a-003e37552c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:45.728164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b694b31-b39b-406c-bc9a-003e37552c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:45.728697 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b694b31-b39b-406c-bc9a-003e37552c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:45.729397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b694b31-b39b-406c-bc9a-003e37552c96 admin admin] PATH_INFO: `/v3/users/4a6369859b1f4cac9eeeec02d6d04a97` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:45.731466 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b694b31-b39b-406c-bc9a-003e37552c96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a6369859b1f4cac9eeeec02d6d04a97)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:45.736963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b694b31-b39b-406c-bc9a-003e37552c96 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:45.840687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9b694b31-b39b-406c-bc9a-003e37552c96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a6369859b1f4cac9eeeec02d6d04a97'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:45.842406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9b694b31-b39b-406c-bc9a-003e37552c96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a6369859b1f4cac9eeeec02d6d04a97'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:45.843676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9b694b31-b39b-406c-bc9a-003e37552c96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a6369859b1f4cac9eeeec02d6d04a97'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:45.853837 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9b694b31-b39b-406c-bc9a-003e37552c96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a6369859b1f4cac9eeeec02d6d04a97'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:45.904793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9b694b31-b39b-406c-bc9a-003e37552c96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a6369859b1f4cac9eeeec02d6d04a97'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:45.950296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1017/2022] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:00:45 2022] DELETE /identity/v3/users/4a6369859b1f4cac9eeeec02d6d04a97 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:46.278011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956e6c1-519e-4604-a85b-263816f2e9ad admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:46.279519 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c956e6c1-519e-4604-a85b-263816f2e9ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:46.296585 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956e6c1-519e-4604-a85b-263816f2e9ad admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:46.362231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bde4b-c770-408a-a31f-74ecdab0466b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:46.363079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bde4b-c770-408a-a31f-74ecdab0466b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:46.363385 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bde4b-c770-408a-a31f-74ecdab0466b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:46.363696 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bde4b-c770-408a-a31f-74ecdab0466b admin admin] PATH_INFO: `/v3/projects/3e09543a93834a7f823c809c8e2ccbd4` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:46.365001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560bde4b-c770-408a-a31f-74ecdab0466b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e09543a93834a7f823c809c8e2ccbd4)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:46.367087 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560bde4b-c770-408a-a31f-74ecdab0466b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:46.424271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-560bde4b-c770-408a-a31f-74ecdab0466b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e09543a93834a7f823c809c8e2ccbd4', 'request_id': 'req-560bde4b-c770-408a-a31f-74ecdab0466b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:46.433633 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-560bde4b-c770-408a-a31f-74ecdab0466b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e09543a93834a7f823c809c8e2ccbd4', 'request_id': 'req-560bde4b-c770-408a-a31f-74ecdab0466b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:46.441296 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-560bde4b-c770-408a-a31f-74ecdab0466b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e09543a93834a7f823c809c8e2ccbd4', 'request_id': 'req-560bde4b-c770-408a-a31f-74ecdab0466b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:46.446105 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:46.456392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-560bde4b-c770-408a-a31f-74ecdab0466b admin admin] The token cache is being invalidate because project 3e09543a93834a7f823c809c8e2ccbd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:46.456919 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-560bde4b-c770-408a-a31f-74ecdab0466b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:46.457668 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-560bde4b-c770-408a-a31f-74ecdab0466b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:46.460348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1006/2023] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:46 2022] DELETE /identity/v3/projects/3e09543a93834a7f823c809c8e2ccbd4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:46.466130 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3796cae-e192-49ff-aae6-e580c95f5cfc None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:46.466425 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3796cae-e192-49ff-aae6-e580c95f5cfc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:46.467362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3796cae-e192-49ff-aae6-e580c95f5cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:46.486937 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c3796cae-e192-49ff-aae6-e580c95f5cfc None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:46.488152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c3796cae-e192-49ff-aae6-e580c95f5cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:46.564583 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1018/2024] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:46.573611 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217c830-a680-4df0-8aa0-4bd50ec79de9 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:46.573906 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217c830-a680-4df0-8aa0-4bd50ec79de9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:46.574193 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217c830-a680-4df0-8aa0-4bd50ec79de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:46.594487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2217c830-a680-4df0-8aa0-4bd50ec79de9 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:46.596005 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2217c830-a680-4df0-8aa0-4bd50ec79de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:46.605419 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1007/2025] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:46.609755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3796cae-e192-49ff-aae6-e580c95f5cfc None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:46.622778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3796cae-e192-49ff-aae6-e580c95f5cfc None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:46.628392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603413d9-7b06-4c82-8f01-b0a3163b61bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:46.629514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603413d9-7b06-4c82-8f01-b0a3163b61bb admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:46.629882 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603413d9-7b06-4c82-8f01-b0a3163b61bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:46.630255 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603413d9-7b06-4c82-8f01-b0a3163b61bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:46.631167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603413d9-7b06-4c82-8f01-b0a3163b61bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:00:46.631549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603413d9-7b06-4c82-8f01-b0a3163b61bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:46.633364 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603413d9-7b06-4c82-8f01-b0a3163b61bb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:46.650554 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1019/2026] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:00:46.663882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217c830-a680-4df0-8aa0-4bd50ec79de9 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:46.676737 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217c830-a680-4df0-8aa0-4bd50ec79de9 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:46.682370 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc221cb-ae6b-4e49-acf7-f0cb2a8da4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:46.683516 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc221cb-ae6b-4e49-acf7-f0cb2a8da4b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:46.683631 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc221cb-ae6b-4e49-acf7-f0cb2a8da4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:46.684145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc221cb-ae6b-4e49-acf7-f0cb2a8da4b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:46.684711 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc221cb-ae6b-4e49-acf7-f0cb2a8da4b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:46.686976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc221cb-ae6b-4e49-acf7-f0cb2a8da4b8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:46.722307 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1008/2027] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:46 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:46.726397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603413d9-7b06-4c82-8f01-b0a3163b61bb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:46.734173 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:46.738126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603413d9-7b06-4c82-8f01-b0a3163b61bb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:46.807203 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec4437-100f-497e-97de-c0cf6626b143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:46.807820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec4437-100f-497e-97de-c0cf6626b143 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:46.808105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec4437-100f-497e-97de-c0cf6626b143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:46.808380 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec4437-100f-497e-97de-c0cf6626b143 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:46.809065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec4437-100f-497e-97de-c0cf6626b143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:46.811782 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec4437-100f-497e-97de-c0cf6626b143 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:46.833610 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1020/2028] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:46 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:00:46.837648 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc221cb-ae6b-4e49-acf7-f0cb2a8da4b8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:46.849796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc221cb-ae6b-4e49-acf7-f0cb2a8da4b8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:46.854987 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335bae7-048b-4a93-83db-2ba11144e4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:46.855682 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335bae7-048b-4a93-83db-2ba11144e4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:46.855974 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335bae7-048b-4a93-83db-2ba11144e4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:46.856255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335bae7-048b-4a93-83db-2ba11144e4f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:46.856989 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335bae7-048b-4a93-83db-2ba11144e4f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:46.859614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335bae7-048b-4a93-83db-2ba11144e4f2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:46.871197 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1009/2029] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:46.875296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec4437-100f-497e-97de-c0cf6626b143 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:46.885490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec4437-100f-497e-97de-c0cf6626b143 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:46.890378 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad99a4-0098-4715-820c-a92779ad22f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:46.891067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad99a4-0098-4715-820c-a92779ad22f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:46.891350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad99a4-0098-4715-820c-a92779ad22f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:46.891624 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad99a4-0098-4715-820c-a92779ad22f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:46.892373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad99a4-0098-4715-820c-a92779ad22f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:46.893956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad99a4-0098-4715-820c-a92779ad22f9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:46.905642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1021/2030] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:46.909795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335bae7-048b-4a93-83db-2ba11144e4f2 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:46.920590 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335bae7-048b-4a93-83db-2ba11144e4f2 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:46.925565 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1c5d7-2f18-407e-96b2-b6fa234986b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:46.926449 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1c5d7-2f18-407e-96b2-b6fa234986b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:46.926766 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1c5d7-2f18-407e-96b2-b6fa234986b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:46.927071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1c5d7-2f18-407e-96b2-b6fa234986b9 admin admin] PATH_INFO: `/v3/projects/62eb17e0c2a14091a95570ea49346c77/users/1e081161247b48418caa6652e7ee00a4/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:46.942242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1c5d7-2f18-407e-96b2-b6fa234986b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62eb17e0c2a14091a95570ea49346c77, user_id=1e081161247b48418caa6652e7ee00a4, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:46.945461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1c5d7-2f18-407e-96b2-b6fa234986b9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:46.961439 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1010/2031] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:46 2022] PUT /identity/v3/projects/62eb17e0c2a14091a95570ea49346c77/users/1e081161247b48418caa6652e7ee00a4/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:46.972372 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb35732-bfa1-4208-9e62-deb43fb47935 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:46.972696 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb35732-bfa1-4208-9e62-deb43fb47935 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:46.972975 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb35732-bfa1-4208-9e62-deb43fb47935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:47.013401 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-dbb35732-bfa1-4208-9e62-deb43fb47935 None None] MFA Rules not processed for user `1e081161247b48418caa6652e7ee00a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:47.015088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-dbb35732-bfa1-4208-9e62-deb43fb47935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:47.089106 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1022/2032] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:47.094314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1c5d7-2f18-407e-96b2-b6fa234986b9 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:47.158613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1c5d7-2f18-407e-96b2-b6fa234986b9 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:47.163584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f49c5-31a2-42f2-b63c-299ac1001fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:47.164267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f49c5-31a2-42f2-b63c-299ac1001fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:47.164547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f49c5-31a2-42f2-b63c-299ac1001fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:47.164825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f49c5-31a2-42f2-b63c-299ac1001fa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:47.165537 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f49c5-31a2-42f2-b63c-299ac1001fa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:47.167473 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f49c5-31a2-42f2-b63c-299ac1001fa7 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:47.193167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1011/2033] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:47 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:47.197200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb35732-bfa1-4208-9e62-deb43fb47935 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:47.208607 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb35732-bfa1-4208-9e62-deb43fb47935 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:47.269133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b1bf9-9932-4ba5-a012-dd56bbf5d4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:47.269821 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b1bf9-9932-4ba5-a012-dd56bbf5d4e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:47.270112 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b1bf9-9932-4ba5-a012-dd56bbf5d4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:47.270388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b1bf9-9932-4ba5-a012-dd56bbf5d4e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:47.271123 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b1bf9-9932-4ba5-a012-dd56bbf5d4e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:47.273044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b1bf9-9932-4ba5-a012-dd56bbf5d4e9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:47.293712 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1023/2034] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:47 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:47.300003 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f49c5-31a2-42f2-b63c-299ac1001fa7 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:47.311584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f49c5-31a2-42f2-b63c-299ac1001fa7 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:47.315924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85a0ad-0c9c-4c27-baab-25525960a112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:47.316578 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85a0ad-0c9c-4c27-baab-25525960a112 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:47.316865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85a0ad-0c9c-4c27-baab-25525960a112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:47.317146 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85a0ad-0c9c-4c27-baab-25525960a112 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:47.317887 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85a0ad-0c9c-4c27-baab-25525960a112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:47.319523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85a0ad-0c9c-4c27-baab-25525960a112 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:47.330160 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1012/2035] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:47.334183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b1bf9-9932-4ba5-a012-dd56bbf5d4e9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:47.339393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:47.346107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b1bf9-9932-4ba5-a012-dd56bbf5d4e9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:47.351235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0b8d6-ee7a-4602-8212-ecad2b483f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:47.352121 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0b8d6-ee7a-4602-8212-ecad2b483f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:47.352405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0b8d6-ee7a-4602-8212-ecad2b483f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:47.352680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0b8d6-ee7a-4602-8212-ecad2b483f34 admin admin] PATH_INFO: `/v3/projects/8524bad6cbda4731a7dd50b5c7d653f0/users/01cea63f7e1a4d518cd180038740e7cf/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:47.368120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f0b8d6-ee7a-4602-8212-ecad2b483f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8524bad6cbda4731a7dd50b5c7d653f0, user_id=01cea63f7e1a4d518cd180038740e7cf, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:47.370826 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f0b8d6-ee7a-4602-8212-ecad2b483f34 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:47.384394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1024/2036] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:47 2022] PUT /identity/v3/projects/8524bad6cbda4731a7dd50b5c7d653f0/users/01cea63f7e1a4d518cd180038740e7cf/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:47.388596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85a0ad-0c9c-4c27-baab-25525960a112 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:47.399188 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85a0ad-0c9c-4c27-baab-25525960a112 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:47.464130 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12557a6-d369-4858-80f7-9bb27ee15eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:47.464820 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12557a6-d369-4858-80f7-9bb27ee15eec admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:47.465113 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12557a6-d369-4858-80f7-9bb27ee15eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:47.465384 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12557a6-d369-4858-80f7-9bb27ee15eec admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:47.466102 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12557a6-d369-4858-80f7-9bb27ee15eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:47.467772 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12557a6-d369-4858-80f7-9bb27ee15eec admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:47.473773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:47.479584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1013/2037] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:47.484347 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0b8d6-ee7a-4602-8212-ecad2b483f34 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:47.495205 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0b8d6-ee7a-4602-8212-ecad2b483f34 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:47.500789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41031a-6302-4ecb-b913-71262bd1daa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:47.501671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41031a-6302-4ecb-b913-71262bd1daa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:47.501950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41031a-6302-4ecb-b913-71262bd1daa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:47.502219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41031a-6302-4ecb-b913-71262bd1daa2 admin admin] PATH_INFO: `/v3/projects/8524bad6cbda4731a7dd50b5c7d653f0/users/01cea63f7e1a4d518cd180038740e7cf/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:47.504197 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41031a-6302-4ecb-b913-71262bd1daa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8524bad6cbda4731a7dd50b5c7d653f0, user_id=01cea63f7e1a4d518cd180038740e7cf, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:47.506870 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41031a-6302-4ecb-b913-71262bd1daa2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:47.517461 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1025/2038] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:47 2022] PUT /identity/v3/projects/8524bad6cbda4731a7dd50b5c7d653f0/users/01cea63f7e1a4d518cd180038740e7cf/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:00:47.528294 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c1cb1-34aa-4dee-a320-7f789c1b5a9a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:47.528592 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c1cb1-34aa-4dee-a320-7f789c1b5a9a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:47.528888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c1cb1-34aa-4dee-a320-7f789c1b5a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:47.559098 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4b3c1cb1-34aa-4dee-a320-7f789c1b5a9a None None] MFA Rules not processed for user `01cea63f7e1a4d518cd180038740e7cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:47.561029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4b3c1cb1-34aa-4dee-a320-7f789c1b5a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:47.632331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1014/2039] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:47.653004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41031a-6302-4ecb-b913-71262bd1daa2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:47.655437 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2c41031a-6302-4ecb-b913-71262bd1daa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:47.719604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41031a-6302-4ecb-b913-71262bd1daa2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:47.723519 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e296e2-745d-4bc8-846f-3865b7bb02de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:47.724276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e296e2-745d-4bc8-846f-3865b7bb02de service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:47.724562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e296e2-745d-4bc8-846f-3865b7bb02de service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:47.724840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e296e2-745d-4bc8-846f-3865b7bb02de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:47.735422 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e296e2-745d-4bc8-846f-3865b7bb02de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:47.736911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e296e2-745d-4bc8-846f-3865b7bb02de service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:47.748027 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1026/2040] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:00:48.173894 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c1cb1-34aa-4dee-a320-7f789c1b5a9a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:48.239836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c1cb1-34aa-4dee-a320-7f789c1b5a9a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:48.244575 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a05c11-0eec-4d2d-80c8-240491efc42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:48.245539 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a05c11-0eec-4d2d-80c8-240491efc42f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:48.246409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a05c11-0eec-4d2d-80c8-240491efc42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:48.246409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a05c11-0eec-4d2d-80c8-240491efc42f admin admin] PATH_INFO: `/v3/users/1e081161247b48418caa6652e7ee00a4` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:48.247417 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a05c11-0eec-4d2d-80c8-240491efc42f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e081161247b48418caa6652e7ee00a4)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:48.250203 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a05c11-0eec-4d2d-80c8-240491efc42f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:48.303482 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-40a05c11-0eec-4d2d-80c8-240491efc42f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e081161247b48418caa6652e7ee00a4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:48.304149 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-40a05c11-0eec-4d2d-80c8-240491efc42f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e081161247b48418caa6652e7ee00a4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:48.312727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-40a05c11-0eec-4d2d-80c8-240491efc42f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e081161247b48418caa6652e7ee00a4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:48.325609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-40a05c11-0eec-4d2d-80c8-240491efc42f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e081161247b48418caa6652e7ee00a4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:48.326861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-40a05c11-0eec-4d2d-80c8-240491efc42f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e081161247b48418caa6652e7ee00a4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:48.349244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1015/2041] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:00:48 2022] DELETE /identity/v3/users/1e081161247b48418caa6652e7ee00a4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:48.353998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d2095-4965-4fe9-89cb-7cd63085793d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:48.354290 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d2095-4965-4fe9-89cb-7cd63085793d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:48.354589 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d2095-4965-4fe9-89cb-7cd63085793d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:48.372240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9a7d2095-4965-4fe9-89cb-7cd63085793d None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:48.374226 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9a7d2095-4965-4fe9-89cb-7cd63085793d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:48.433483 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1027/2042] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:48.443908 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a05c11-0eec-4d2d-80c8-240491efc42f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:48.445400 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-40a05c11-0eec-4d2d-80c8-240491efc42f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:48.453977 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:48.513288 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a05c11-0eec-4d2d-80c8-240491efc42f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:48.518852 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da174bdb-77b2-4734-924e-56cab73cca84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:48.519515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da174bdb-77b2-4734-924e-56cab73cca84 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:48.519797 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da174bdb-77b2-4734-924e-56cab73cca84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:48.520074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da174bdb-77b2-4734-924e-56cab73cca84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:48.530859 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da174bdb-77b2-4734-924e-56cab73cca84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:48.532085 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da174bdb-77b2-4734-924e-56cab73cca84 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:48.542306 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1016/2043] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:00:48.820769 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d2095-4965-4fe9-89cb-7cd63085793d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:48.822453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9a7d2095-4965-4fe9-89cb-7cd63085793d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:48.834877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d2095-4965-4fe9-89cb-7cd63085793d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:48.840275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc980d43-91cc-4060-86ba-6c9ac24c4390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:48.841435 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc980d43-91cc-4060-86ba-6c9ac24c4390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:48.841767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc980d43-91cc-4060-86ba-6c9ac24c4390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:48.842091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc980d43-91cc-4060-86ba-6c9ac24c4390 admin admin] PATH_INFO: `/v3/projects/62eb17e0c2a14091a95570ea49346c77` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:48.843655 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc980d43-91cc-4060-86ba-6c9ac24c4390 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62eb17e0c2a14091a95570ea49346c77)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:48.853294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc980d43-91cc-4060-86ba-6c9ac24c4390 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:48.900453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-fc980d43-91cc-4060-86ba-6c9ac24c4390 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62eb17e0c2a14091a95570ea49346c77', 'request_id': 'req-fc980d43-91cc-4060-86ba-6c9ac24c4390'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:48.908556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-fc980d43-91cc-4060-86ba-6c9ac24c4390 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62eb17e0c2a14091a95570ea49346c77', 'request_id': 'req-fc980d43-91cc-4060-86ba-6c9ac24c4390'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:48.915173 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-fc980d43-91cc-4060-86ba-6c9ac24c4390 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62eb17e0c2a14091a95570ea49346c77', 'request_id': 'req-fc980d43-91cc-4060-86ba-6c9ac24c4390'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:48.917412 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:48.929629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-fc980d43-91cc-4060-86ba-6c9ac24c4390 admin admin] The token cache is being invalidate because project 62eb17e0c2a14091a95570ea49346c77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:48.929921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-fc980d43-91cc-4060-86ba-6c9ac24c4390 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:48.930624 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-fc980d43-91cc-4060-86ba-6c9ac24c4390 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:48.932222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1028/2044] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:48 2022] DELETE /identity/v3/projects/62eb17e0c2a14091a95570ea49346c77 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:48.937119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da174bdb-77b2-4734-924e-56cab73cca84 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:48.947408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da174bdb-77b2-4734-924e-56cab73cca84 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:49.011902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee73ef-b80d-422f-b3c9-cd5cd7c1b1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:49.012852 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee73ef-b80d-422f-b3c9-cd5cd7c1b1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:49.013170 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee73ef-b80d-422f-b3c9-cd5cd7c1b1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:49.013512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee73ef-b80d-422f-b3c9-cd5cd7c1b1b2 admin admin] PATH_INFO: `/v3/users/01cea63f7e1a4d518cd180038740e7cf` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:49.015564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee73ef-b80d-422f-b3c9-cd5cd7c1b1b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01cea63f7e1a4d518cd180038740e7cf)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:49.017980 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee73ef-b80d-422f-b3c9-cd5cd7c1b1b2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:49.069378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6fee73ef-b80d-422f-b3c9-cd5cd7c1b1b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01cea63f7e1a4d518cd180038740e7cf'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:49.070076 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6fee73ef-b80d-422f-b3c9-cd5cd7c1b1b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01cea63f7e1a4d518cd180038740e7cf'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:49.077918 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6fee73ef-b80d-422f-b3c9-cd5cd7c1b1b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01cea63f7e1a4d518cd180038740e7cf'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:49.090239 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6fee73ef-b80d-422f-b3c9-cd5cd7c1b1b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01cea63f7e1a4d518cd180038740e7cf'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:49.091127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6fee73ef-b80d-422f-b3c9-cd5cd7c1b1b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01cea63f7e1a4d518cd180038740e7cf'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:49.106367 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1017/2045] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:00:48 2022] DELETE /identity/v3/users/01cea63f7e1a4d518cd180038740e7cf => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:49.395014 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc980d43-91cc-4060-86ba-6c9ac24c4390 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:49.396438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-fc980d43-91cc-4060-86ba-6c9ac24c4390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:49.411620 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc980d43-91cc-4060-86ba-6c9ac24c4390 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:49.483658 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1af8fa-b91f-42d7-b9e7-64cef31b10f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:49.484811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1af8fa-b91f-42d7-b9e7-64cef31b10f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:49.485103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1af8fa-b91f-42d7-b9e7-64cef31b10f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:49.485387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1af8fa-b91f-42d7-b9e7-64cef31b10f5 admin admin] PATH_INFO: `/v3/projects/8524bad6cbda4731a7dd50b5c7d653f0` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:49.486931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1af8fa-b91f-42d7-b9e7-64cef31b10f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8524bad6cbda4731a7dd50b5c7d653f0)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:49.488960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1af8fa-b91f-42d7-b9e7-64cef31b10f5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:49.539078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8e1af8fa-b91f-42d7-b9e7-64cef31b10f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8524bad6cbda4731a7dd50b5c7d653f0', 'request_id': 'req-8e1af8fa-b91f-42d7-b9e7-64cef31b10f5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:49.546976 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8e1af8fa-b91f-42d7-b9e7-64cef31b10f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8524bad6cbda4731a7dd50b5c7d653f0', 'request_id': 'req-8e1af8fa-b91f-42d7-b9e7-64cef31b10f5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:49.553577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8e1af8fa-b91f-42d7-b9e7-64cef31b10f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8524bad6cbda4731a7dd50b5c7d653f0', 'request_id': 'req-8e1af8fa-b91f-42d7-b9e7-64cef31b10f5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:49.565346 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8e1af8fa-b91f-42d7-b9e7-64cef31b10f5 admin admin] The token cache is being invalidate because project 8524bad6cbda4731a7dd50b5c7d653f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:49.565641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8e1af8fa-b91f-42d7-b9e7-64cef31b10f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:49.566283 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8e1af8fa-b91f-42d7-b9e7-64cef31b10f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:49.567825 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1029/2046] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:49 2022] DELETE /identity/v3/projects/8524bad6cbda4731a7dd50b5c7d653f0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:49.573486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4add9-efd3-429f-97a7-56dd7b8fc643 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:49.574014 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4add9-efd3-429f-97a7-56dd7b8fc643 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:49.574213 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4add9-efd3-429f-97a7-56dd7b8fc643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:49.591898 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f1c4add9-efd3-429f-97a7-56dd7b8fc643 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:49.593105 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f1c4add9-efd3-429f-97a7-56dd7b8fc643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:49.617013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:49.659111 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1018/2047] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:49.667223 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5ad18-4551-413e-b508-f4157ad22468 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:49.667545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5ad18-4551-413e-b508-f4157ad22468 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:49.667836 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5ad18-4551-413e-b508-f4157ad22468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:49.686539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2af5ad18-4551-413e-b508-f4157ad22468 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:49.687776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2af5ad18-4551-413e-b508-f4157ad22468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:49.695771 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1030/2048] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:49.699915 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4add9-efd3-429f-97a7-56dd7b8fc643 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:49.711021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4add9-efd3-429f-97a7-56dd7b8fc643 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:49.715962 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9529dd-de5d-4c96-830d-e7657bead515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:49.716744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9529dd-de5d-4c96-830d-e7657bead515 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:49.717034 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9529dd-de5d-4c96-830d-e7657bead515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:49.717398 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9529dd-de5d-4c96-830d-e7657bead515 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:49.718676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9529dd-de5d-4c96-830d-e7657bead515 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:00:49.718988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9529dd-de5d-4c96-830d-e7657bead515 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:49.720590 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9529dd-de5d-4c96-830d-e7657bead515 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:49.733416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1019/2049] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:00:49.737707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af5ad18-4551-413e-b508-f4157ad22468 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:49.747188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af5ad18-4551-413e-b508-f4157ad22468 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:49.753319 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ffe02c-ef7f-4556-905a-95bbfef08892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:49.753968 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffe02c-ef7f-4556-905a-95bbfef08892 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:49.754253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffe02c-ef7f-4556-905a-95bbfef08892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:49.754533 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffe02c-ef7f-4556-905a-95bbfef08892 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:49.755247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ffe02c-ef7f-4556-905a-95bbfef08892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:49.757136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ffe02c-ef7f-4556-905a-95bbfef08892 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:49.785700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1031/2050] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:49 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:00:49.789680 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9529dd-de5d-4c96-830d-e7657bead515 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:49.800643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9529dd-de5d-4c96-830d-e7657bead515 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:49.874631 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e25c7-3d19-4a0f-aaa6-099619ee37ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:49.875359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e25c7-3d19-4a0f-aaa6-099619ee37ef admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:49.875787 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e25c7-3d19-4a0f-aaa6-099619ee37ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:49.876092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e25c7-3d19-4a0f-aaa6-099619ee37ef admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:49.877125 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e25c7-3d19-4a0f-aaa6-099619ee37ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:49.879374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e25c7-3d19-4a0f-aaa6-099619ee37ef admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:49.913041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1020/2051] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:49 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:49.917381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ffe02c-ef7f-4556-905a-95bbfef08892 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:49.937707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ffe02c-ef7f-4556-905a-95bbfef08892 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:49.942526 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0ce84-34f3-4115-b4fa-970c9432f3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:49.949299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0ce84-34f3-4115-b4fa-970c9432f3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:49.949594 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0ce84-34f3-4115-b4fa-970c9432f3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:49.949888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0ce84-34f3-4115-b4fa-970c9432f3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:49.950780 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0ce84-34f3-4115-b4fa-970c9432f3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:49.954044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0ce84-34f3-4115-b4fa-970c9432f3b7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:49.972867 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1032/2052] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:49.976844 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e25c7-3d19-4a0f-aaa6-099619ee37ef admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:49.988244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e25c7-3d19-4a0f-aaa6-099619ee37ef admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:49.995761 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe572e-2ae8-4278-9c37-336cfd6f0e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:49.996916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe572e-2ae8-4278-9c37-336cfd6f0e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:49.996916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe572e-2ae8-4278-9c37-336cfd6f0e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:49.997163 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe572e-2ae8-4278-9c37-336cfd6f0e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:49.998870 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe572e-2ae8-4278-9c37-336cfd6f0e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:50.000542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe572e-2ae8-4278-9c37-336cfd6f0e90 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:50.011581 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1021/2053] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:50.015788 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0ce84-34f3-4115-b4fa-970c9432f3b7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:50.025759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0ce84-34f3-4115-b4fa-970c9432f3b7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:50.031412 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201f045-3c0a-4a33-a66f-a6ee7c4df0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:50.032292 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201f045-3c0a-4a33-a66f-a6ee7c4df0f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:50.032576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201f045-3c0a-4a33-a66f-a6ee7c4df0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:50.032964 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201f045-3c0a-4a33-a66f-a6ee7c4df0f5 admin admin] PATH_INFO: `/v3/projects/1ea3838f630b40b4bc6dfe785a34816d/users/7c75cc39ffb34dcbb99237e24e610163/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:50.051833 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201f045-3c0a-4a33-a66f-a6ee7c4df0f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ea3838f630b40b4bc6dfe785a34816d, user_id=7c75cc39ffb34dcbb99237e24e610163, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:50.055168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201f045-3c0a-4a33-a66f-a6ee7c4df0f5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:50.071474 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1033/2054] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:50 2022] PUT /identity/v3/projects/1ea3838f630b40b4bc6dfe785a34816d/users/7c75cc39ffb34dcbb99237e24e610163/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:50.082297 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbea3e-614f-4abf-80cb-fb272a7c97dd None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:50.082619 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbea3e-614f-4abf-80cb-fb272a7c97dd None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:50.082907 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbea3e-614f-4abf-80cb-fb272a7c97dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:50.114100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-7ebbea3e-614f-4abf-80cb-fb272a7c97dd None None] MFA Rules not processed for user `7c75cc39ffb34dcbb99237e24e610163`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:50.115483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7ebbea3e-614f-4abf-80cb-fb272a7c97dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:50.198234 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1022/2055] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:00:50.210181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201f045-3c0a-4a33-a66f-a6ee7c4df0f5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:50.211918 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5201f045-3c0a-4a33-a66f-a6ee7c4df0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:50.233454 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:50.278067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201f045-3c0a-4a33-a66f-a6ee7c4df0f5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:50.283196 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa8e678-872d-4536-8cfb-cb30507dc1b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:50.283872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8e678-872d-4536-8cfb-cb30507dc1b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:50.284159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8e678-872d-4536-8cfb-cb30507dc1b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:50.284522 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8e678-872d-4536-8cfb-cb30507dc1b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:50.299748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa8e678-872d-4536-8cfb-cb30507dc1b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:50.301227 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa8e678-872d-4536-8cfb-cb30507dc1b9 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:50.311448 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1034/2056] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:00:53.932898 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbea3e-614f-4abf-80cb-fb272a7c97dd None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:53.967252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:54.009711 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbea3e-614f-4abf-80cb-fb272a7c97dd None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:54.015498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d5c8e-7766-4f94-aaad-d9280f5b16dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:54.016384 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d5c8e-7766-4f94-aaad-d9280f5b16dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:54.016672 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d5c8e-7766-4f94-aaad-d9280f5b16dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:54.016951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d5c8e-7766-4f94-aaad-d9280f5b16dd admin admin] PATH_INFO: `/v3/users/7c75cc39ffb34dcbb99237e24e610163` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:54.018218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d5c8e-7766-4f94-aaad-d9280f5b16dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c75cc39ffb34dcbb99237e24e610163)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:54.020297 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d5c8e-7766-4f94-aaad-d9280f5b16dd admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:54.068589 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b61d5c8e-7766-4f94-aaad-d9280f5b16dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c75cc39ffb34dcbb99237e24e610163'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:54.069258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b61d5c8e-7766-4f94-aaad-d9280f5b16dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c75cc39ffb34dcbb99237e24e610163'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:54.077040 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b61d5c8e-7766-4f94-aaad-d9280f5b16dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c75cc39ffb34dcbb99237e24e610163'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:54.090203 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b61d5c8e-7766-4f94-aaad-d9280f5b16dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c75cc39ffb34dcbb99237e24e610163'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:54.090984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b61d5c8e-7766-4f94-aaad-d9280f5b16dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c75cc39ffb34dcbb99237e24e610163'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:54.108654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1023/2057] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:00:53 2022] DELETE /identity/v3/users/7c75cc39ffb34dcbb99237e24e610163 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:54.110246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e16ba-6eac-466d-8a0c-8a828fe79edd None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:54.110806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e16ba-6eac-466d-8a0c-8a828fe79edd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:54.111791 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e16ba-6eac-466d-8a0c-8a828fe79edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:54.131613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b04e16ba-6eac-466d-8a0c-8a828fe79edd None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:54.132843 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b04e16ba-6eac-466d-8a0c-8a828fe79edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:54.206187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1035/2058] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:54.216206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d5c8e-7766-4f94-aaad-d9280f5b16dd admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:54.217757 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b61d5c8e-7766-4f94-aaad-d9280f5b16dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:54.280003 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d5c8e-7766-4f94-aaad-d9280f5b16dd admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:54.283751 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90fa81-ce8c-4cac-92e8-6bc43e8afe1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:54.284430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90fa81-ce8c-4cac-92e8-6bc43e8afe1e service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:54.284706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90fa81-ce8c-4cac-92e8-6bc43e8afe1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:54.284979 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90fa81-ce8c-4cac-92e8-6bc43e8afe1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:54.293979 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab90fa81-ce8c-4cac-92e8-6bc43e8afe1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:54.295566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab90fa81-ce8c-4cac-92e8-6bc43e8afe1e service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:54.304942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1024/2059] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:00:54.612432 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e16ba-6eac-466d-8a0c-8a828fe79edd None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:54.613963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b04e16ba-6eac-466d-8a0c-8a828fe79edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:54.638844 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e16ba-6eac-466d-8a0c-8a828fe79edd None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:54.646384 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f8e85-f389-438a-acd6-8bd1fcafbe8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:54.647960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f8e85-f389-438a-acd6-8bd1fcafbe8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:54.648257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f8e85-f389-438a-acd6-8bd1fcafbe8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:54.648540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f8e85-f389-438a-acd6-8bd1fcafbe8a admin admin] PATH_INFO: `/v3/projects/1ea3838f630b40b4bc6dfe785a34816d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:54.650695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f8e85-f389-438a-acd6-8bd1fcafbe8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ea3838f630b40b4bc6dfe785a34816d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:54.653104 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f8e85-f389-438a-acd6-8bd1fcafbe8a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:54.673929 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90fa81-ce8c-4cac-92e8-6bc43e8afe1e service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:54.675903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ab90fa81-ce8c-4cac-92e8-6bc43e8afe1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:54.692372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90fa81-ce8c-4cac-92e8-6bc43e8afe1e service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:54.697243 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030dda0-9942-416e-aa1c-c58d8c2c2a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:54.698487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030dda0-9942-416e-aa1c-c58d8c2c2a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:54.698811 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030dda0-9942-416e-aa1c-c58d8c2c2a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:54.699089 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030dda0-9942-416e-aa1c-c58d8c2c2a0d admin admin] PATH_INFO: `/v3/users/ab77c441b2004df0a6556b4c7c0efa95` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:54.700442 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030dda0-9942-416e-aa1c-c58d8c2c2a0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab77c441b2004df0a6556b4c7c0efa95)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:54.703658 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030dda0-9942-416e-aa1c-c58d8c2c2a0d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:54.724770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ac2f8e85-f389-438a-acd6-8bd1fcafbe8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ea3838f630b40b4bc6dfe785a34816d', 'request_id': 'req-ac2f8e85-f389-438a-acd6-8bd1fcafbe8a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:54.743010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ac2f8e85-f389-438a-acd6-8bd1fcafbe8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ea3838f630b40b4bc6dfe785a34816d', 'request_id': 'req-ac2f8e85-f389-438a-acd6-8bd1fcafbe8a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:54.751119 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ac2f8e85-f389-438a-acd6-8bd1fcafbe8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ea3838f630b40b4bc6dfe785a34816d', 'request_id': 'req-ac2f8e85-f389-438a-acd6-8bd1fcafbe8a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:54.764659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ac2f8e85-f389-438a-acd6-8bd1fcafbe8a admin admin] The token cache is being invalidate because project 1ea3838f630b40b4bc6dfe785a34816d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:54.765136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ac2f8e85-f389-438a-acd6-8bd1fcafbe8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:54.765822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ac2f8e85-f389-438a-acd6-8bd1fcafbe8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:54.767565 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1036/2060] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:54 2022] DELETE /identity/v3/projects/1ea3838f630b40b4bc6dfe785a34816d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:54.771046 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:54.772418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c8ec4-e75c-4246-b895-1ad551a78636 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:54.772955 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c8ec4-e75c-4246-b895-1ad551a78636 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:54.773384 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c8ec4-e75c-4246-b895-1ad551a78636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:54.787970 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b030dda0-9942-416e-aa1c-c58d8c2c2a0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab77c441b2004df0a6556b4c7c0efa95'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:54.789564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b030dda0-9942-416e-aa1c-c58d8c2c2a0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab77c441b2004df0a6556b4c7c0efa95'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:54.799842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-611c8ec4-e75c-4246-b895-1ad551a78636 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:54.801119 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-611c8ec4-e75c-4246-b895-1ad551a78636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:54.803025 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b030dda0-9942-416e-aa1c-c58d8c2c2a0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab77c441b2004df0a6556b4c7c0efa95'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:54.833959 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b030dda0-9942-416e-aa1c-c58d8c2c2a0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab77c441b2004df0a6556b4c7c0efa95'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:54.835004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b030dda0-9942-416e-aa1c-c58d8c2c2a0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab77c441b2004df0a6556b4c7c0efa95'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:54.853792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1025/2061] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:00:54 2022] DELETE /identity/v3/users/ab77c441b2004df0a6556b4c7c0efa95 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:00:54.858990 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f82736-79f0-4d21-b3c3-7fdf3a91bf8a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:54.859289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f82736-79f0-4d21-b3c3-7fdf3a91bf8a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:54.859570 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f82736-79f0-4d21-b3c3-7fdf3a91bf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:54.869045 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:54.881776 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-58f82736-79f0-4d21-b3c3-7fdf3a91bf8a None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:54.883173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-58f82736-79f0-4d21-b3c3-7fdf3a91bf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:54.897520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1037/2062] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:00:54.906924 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7463c-25f9-4d47-b7ca-1f37eac96e5a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:54.907570 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7463c-25f9-4d47-b7ca-1f37eac96e5a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:54.907857 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7463c-25f9-4d47-b7ca-1f37eac96e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:54.928399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-94a7463c-25f9-4d47-b7ca-1f37eac96e5a None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:54.929564 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-94a7463c-25f9-4d47-b7ca-1f37eac96e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:54.954296 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1026/2063] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:00:54.967781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f82736-79f0-4d21-b3c3-7fdf3a91bf8a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:54.969249 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-58f82736-79f0-4d21-b3c3-7fdf3a91bf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:55.013593 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1038/2064] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:00:55.018010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7463c-25f9-4d47-b7ca-1f37eac96e5a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:55.027648 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7463c-25f9-4d47-b7ca-1f37eac96e5a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:55.034322 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735ae0e-77a2-4c77-b656-4660128ed424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:55.035007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735ae0e-77a2-4c77-b656-4660128ed424 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:55.035267 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735ae0e-77a2-4c77-b656-4660128ed424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:55.035545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735ae0e-77a2-4c77-b656-4660128ed424 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:55.035708 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f82736-79f0-4d21-b3c3-7fdf3a91bf8a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:55.036359 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735ae0e-77a2-4c77-b656-4660128ed424 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:00:55.036605 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735ae0e-77a2-4c77-b656-4660128ed424 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:55.038243 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735ae0e-77a2-4c77-b656-4660128ed424 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:55.041002 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28f1fc-2e00-4625-844f-126f822bab28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:55.041698 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28f1fc-2e00-4625-844f-126f822bab28 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:55.042004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28f1fc-2e00-4625-844f-126f822bab28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:55.042283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28f1fc-2e00-4625-844f-126f822bab28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:55.054754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1039/2065] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:00:55.058624 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735ae0e-77a2-4c77-b656-4660128ed424 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:55.061283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28f1fc-2e00-4625-844f-126f822bab28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:55.063063 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28f1fc-2e00-4625-844f-126f822bab28 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:55.071341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735ae0e-77a2-4c77-b656-4660128ed424 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:55.075110 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1027/2066] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:00:55.077256 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfe4e7-cc61-4596-9017-85c094d622b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:55.077982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfe4e7-cc61-4596-9017-85c094d622b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:55.078251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfe4e7-cc61-4596-9017-85c094d622b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:55.078517 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfe4e7-cc61-4596-9017-85c094d622b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:55.079257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfe4e7-cc61-4596-9017-85c094d622b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:55.081176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfe4e7-cc61-4596-9017-85c094d622b4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:55.113761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1040/2067] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:55 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:55.118083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28f1fc-2e00-4625-844f-126f822bab28 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:55.129804 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28f1fc-2e00-4625-844f-126f822bab28 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:55.228098 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d975d00-0843-4de1-a0fd-421f3e2e7d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:55.228674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d975d00-0843-4de1-a0fd-421f3e2e7d09 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:55.229015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d975d00-0843-4de1-a0fd-421f3e2e7d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:55.229299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d975d00-0843-4de1-a0fd-421f3e2e7d09 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:55.230007 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d975d00-0843-4de1-a0fd-421f3e2e7d09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:55.232893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d975d00-0843-4de1-a0fd-421f3e2e7d09 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:55.240536 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:55.258633 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1028/2068] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:55 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:55.262732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfe4e7-cc61-4596-9017-85c094d622b4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:55.280464 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfe4e7-cc61-4596-9017-85c094d622b4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:55.286031 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357384a3-005f-42ba-a0eb-e9dc75d05b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:55.286955 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357384a3-005f-42ba-a0eb-e9dc75d05b75 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:55.287305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357384a3-005f-42ba-a0eb-e9dc75d05b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:55.287585 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357384a3-005f-42ba-a0eb-e9dc75d05b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:55.288450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357384a3-005f-42ba-a0eb-e9dc75d05b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:55.290606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357384a3-005f-42ba-a0eb-e9dc75d05b75 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:55.308640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1041/2069] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:55.308827 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d975d00-0843-4de1-a0fd-421f3e2e7d09 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:55.321015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d975d00-0843-4de1-a0fd-421f3e2e7d09 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:55.326867 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a9e93-df10-4e17-9a3c-afada51fb927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:55.327725 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a9e93-df10-4e17-9a3c-afada51fb927 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:55.328013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a9e93-df10-4e17-9a3c-afada51fb927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:55.328294 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a9e93-df10-4e17-9a3c-afada51fb927 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:55.329030 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a9e93-df10-4e17-9a3c-afada51fb927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:55.332445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a9e93-df10-4e17-9a3c-afada51fb927 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:55.345067 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1029/2070] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:55.349350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357384a3-005f-42ba-a0eb-e9dc75d05b75 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:55.360333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357384a3-005f-42ba-a0eb-e9dc75d05b75 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:55.365797 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08265ad5-bc18-4b21-9c73-4a673b40ff8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:55.366727 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08265ad5-bc18-4b21-9c73-4a673b40ff8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:55.367009 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08265ad5-bc18-4b21-9c73-4a673b40ff8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:55.367309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08265ad5-bc18-4b21-9c73-4a673b40ff8d admin admin] PATH_INFO: `/v3/projects/741ee13164894b49be1a0fc6c1385a8e/users/047e841cefdd4975947b2c154412953b/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:55.385027 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08265ad5-bc18-4b21-9c73-4a673b40ff8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=741ee13164894b49be1a0fc6c1385a8e, user_id=047e841cefdd4975947b2c154412953b, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:55.387727 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08265ad5-bc18-4b21-9c73-4a673b40ff8d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:55.423676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1042/2071] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:55 2022] PUT /identity/v3/projects/741ee13164894b49be1a0fc6c1385a8e/users/047e841cefdd4975947b2c154412953b/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:55.428927 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c720d55-7817-4a6a-8595-256c04563964 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:55.429243 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c720d55-7817-4a6a-8595-256c04563964 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:55.430001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c720d55-7817-4a6a-8595-256c04563964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:55.451285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4c720d55-7817-4a6a-8595-256c04563964 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:55.453750 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4c720d55-7817-4a6a-8595-256c04563964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:55.524995 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08265ad5-bc18-4b21-9c73-4a673b40ff8d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:55.527819 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-08265ad5-bc18-4b21-9c73-4a673b40ff8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:55.586642 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1030/2072] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:55.598912 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c720d55-7817-4a6a-8595-256c04563964 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:55.634940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08265ad5-bc18-4b21-9c73-4a673b40ff8d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:55.651625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614f25d-3c9c-491e-8849-6e1bfb677a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:55.652540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614f25d-3c9c-491e-8849-6e1bfb677a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:55.652827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614f25d-3c9c-491e-8849-6e1bfb677a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:55.653107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614f25d-3c9c-491e-8849-6e1bfb677a37 admin admin] PATH_INFO: `/v3/projects/16741d513de64934a63abf8e754b1d17` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:55.655550 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614f25d-3c9c-491e-8849-6e1bfb677a37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16741d513de64934a63abf8e754b1d17)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:55.657947 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614f25d-3c9c-491e-8849-6e1bfb677a37 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:55.674614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c720d55-7817-4a6a-8595-256c04563964 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:55.678458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c6930-e4a2-448e-8b41-711a9e7aa0ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:55.679275 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c6930-e4a2-448e-8b41-711a9e7aa0ed service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:55.679624 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c6930-e4a2-448e-8b41-711a9e7aa0ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:55.679967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c6930-e4a2-448e-8b41-711a9e7aa0ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:55.683622 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:55.690959 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c6930-e4a2-448e-8b41-711a9e7aa0ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:55.692649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c6930-e4a2-448e-8b41-711a9e7aa0ed service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:55.705857 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1031/2073] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:00:55.727741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8614f25d-3c9c-491e-8849-6e1bfb677a37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16741d513de64934a63abf8e754b1d17', 'request_id': 'req-8614f25d-3c9c-491e-8849-6e1bfb677a37'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:55.737732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8614f25d-3c9c-491e-8849-6e1bfb677a37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16741d513de64934a63abf8e754b1d17', 'request_id': 'req-8614f25d-3c9c-491e-8849-6e1bfb677a37'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:55.745819 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8614f25d-3c9c-491e-8849-6e1bfb677a37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16741d513de64934a63abf8e754b1d17', 'request_id': 'req-8614f25d-3c9c-491e-8849-6e1bfb677a37'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:55.759423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8614f25d-3c9c-491e-8849-6e1bfb677a37 admin admin] The token cache is being invalidate because project 16741d513de64934a63abf8e754b1d17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:55.759988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8614f25d-3c9c-491e-8849-6e1bfb677a37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:55.761197 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8614f25d-3c9c-491e-8849-6e1bfb677a37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:55.763152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1043/2074] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:55 2022] DELETE /identity/v3/projects/16741d513de64934a63abf8e754b1d17 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:00:55.772716 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668774b5-f2cc-48fd-8241-5526394eb4cc None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:55.772716 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668774b5-f2cc-48fd-8241-5526394eb4cc None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:55.772716 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668774b5-f2cc-48fd-8241-5526394eb4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:55.800001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-668774b5-f2cc-48fd-8241-5526394eb4cc None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:55.801570 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-668774b5-f2cc-48fd-8241-5526394eb4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:55.907764 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1032/2075] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:55.916179 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f762f3-2ecb-4489-a0f0-a8002f9f31b9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:55.916179 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f762f3-2ecb-4489-a0f0-a8002f9f31b9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:55.916531 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f762f3-2ecb-4489-a0f0-a8002f9f31b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:55.940284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-51f762f3-2ecb-4489-a0f0-a8002f9f31b9 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:55.941786 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-51f762f3-2ecb-4489-a0f0-a8002f9f31b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:55.954621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1044/2076] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:55.963477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668774b5-f2cc-48fd-8241-5526394eb4cc None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:55.979847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668774b5-f2cc-48fd-8241-5526394eb4cc None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:55.985273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3b1f7-9f8d-49be-90ff-7232f43169f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:55.985941 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3b1f7-9f8d-49be-90ff-7232f43169f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:55.986220 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3b1f7-9f8d-49be-90ff-7232f43169f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:55.986493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3b1f7-9f8d-49be-90ff-7232f43169f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:55.987253 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b3b1f7-9f8d-49be-90ff-7232f43169f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:00:55.987560 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b3b1f7-9f8d-49be-90ff-7232f43169f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:55.989156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b3b1f7-9f8d-49be-90ff-7232f43169f2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:56.007198 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1033/2077] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:00:56.009491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f762f3-2ecb-4489-a0f0-a8002f9f31b9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:56.020172 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f762f3-2ecb-4489-a0f0-a8002f9f31b9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:56.025259 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89da0b1-6bef-45cb-86be-581de492ba74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:56.026191 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89da0b1-6bef-45cb-86be-581de492ba74 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:56.026478 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89da0b1-6bef-45cb-86be-581de492ba74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:56.026781 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89da0b1-6bef-45cb-86be-581de492ba74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:56.027550 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89da0b1-6bef-45cb-86be-581de492ba74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:56.029869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89da0b1-6bef-45cb-86be-581de492ba74 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:56.057290 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1045/2078] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:56 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:56.062614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3b1f7-9f8d-49be-90ff-7232f43169f2 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:56.073063 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3b1f7-9f8d-49be-90ff-7232f43169f2 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:56.082885 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:56.151884 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff89241-7ab1-4c65-97fa-a9a8db9a7d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:56.152569 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff89241-7ab1-4c65-97fa-a9a8db9a7d14 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:56.154441 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff89241-7ab1-4c65-97fa-a9a8db9a7d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:56.154441 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff89241-7ab1-4c65-97fa-a9a8db9a7d14 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:56.154441 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff89241-7ab1-4c65-97fa-a9a8db9a7d14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:56.156541 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff89241-7ab1-4c65-97fa-a9a8db9a7d14 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:56.180419 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1034/2079] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:56 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:00:56.184894 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89da0b1-6bef-45cb-86be-581de492ba74 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:56.202023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89da0b1-6bef-45cb-86be-581de492ba74 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:56.207626 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3381005-2c5c-4ba7-bdfc-1088150ef5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:56.208318 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3381005-2c5c-4ba7-bdfc-1088150ef5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:56.208751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3381005-2c5c-4ba7-bdfc-1088150ef5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:56.209041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3381005-2c5c-4ba7-bdfc-1088150ef5b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:56.210485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3381005-2c5c-4ba7-bdfc-1088150ef5b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:56.212390 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3381005-2c5c-4ba7-bdfc-1088150ef5b9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:56.226711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1046/2080] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:56.231212 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff89241-7ab1-4c65-97fa-a9a8db9a7d14 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:56.248627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff89241-7ab1-4c65-97fa-a9a8db9a7d14 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:56.255075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191db394-b9d5-4447-9637-09fc6cf68923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:56.255927 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191db394-b9d5-4447-9637-09fc6cf68923 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:56.256329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191db394-b9d5-4447-9637-09fc6cf68923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:56.256991 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191db394-b9d5-4447-9637-09fc6cf68923 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:56.258250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191db394-b9d5-4447-9637-09fc6cf68923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:56.260606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191db394-b9d5-4447-9637-09fc6cf68923 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:56.276456 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1035/2081] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:00:56.281535 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3381005-2c5c-4ba7-bdfc-1088150ef5b9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:56.295091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3381005-2c5c-4ba7-bdfc-1088150ef5b9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:56.301443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cf5c5-e557-4db8-b791-fbd5528c2fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:56.302306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cf5c5-e557-4db8-b791-fbd5528c2fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:56.302829 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cf5c5-e557-4db8-b791-fbd5528c2fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:56.303300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cf5c5-e557-4db8-b791-fbd5528c2fd5 admin admin] PATH_INFO: `/v3/projects/a3bd32bf0f224fb8be1bf288e72d6614/users/1001192119c04a1fb09938c41fd0d4f2/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:56.323678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cf5c5-e557-4db8-b791-fbd5528c2fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3bd32bf0f224fb8be1bf288e72d6614, user_id=1001192119c04a1fb09938c41fd0d4f2, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:56.327368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cf5c5-e557-4db8-b791-fbd5528c2fd5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:56.350629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1047/2082] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:56 2022] PUT /identity/v3/projects/a3bd32bf0f224fb8be1bf288e72d6614/users/1001192119c04a1fb09938c41fd0d4f2/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:56.358374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5491c-4270-45b0-a8b5-5ec0a494a3a5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:56.359297 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5491c-4270-45b0-a8b5-5ec0a494a3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:56.359438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5491c-4270-45b0-a8b5-5ec0a494a3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:56.389183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-29a5491c-4270-45b0-a8b5-5ec0a494a3a5 None None] MFA Rules not processed for user `1001192119c04a1fb09938c41fd0d4f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:56.390444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-29a5491c-4270-45b0-a8b5-5ec0a494a3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:56.475961 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1036/2083] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:56.480869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cf5c5-e557-4db8-b791-fbd5528c2fd5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:56.574539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cf5c5-e557-4db8-b791-fbd5528c2fd5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:56.580706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1ffd3-4c62-4c7e-9c12-dac4576a0949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:56.581738 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1ffd3-4c62-4c7e-9c12-dac4576a0949 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:56.582259 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1ffd3-4c62-4c7e-9c12-dac4576a0949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:56.583577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1ffd3-4c62-4c7e-9c12-dac4576a0949 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:56.583814 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1ffd3-4c62-4c7e-9c12-dac4576a0949 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:56.585955 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1ffd3-4c62-4c7e-9c12-dac4576a0949 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:56.622899 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1048/2084] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:56 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:56.627416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5491c-4270-45b0-a8b5-5ec0a494a3a5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:56.647567 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5491c-4270-45b0-a8b5-5ec0a494a3a5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:56.745934 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ed862-3bc6-4553-b1f2-eacbc0a60d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:56.747105 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ed862-3bc6-4553-b1f2-eacbc0a60d6b admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:56.747907 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ed862-3bc6-4553-b1f2-eacbc0a60d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:56.748819 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ed862-3bc6-4553-b1f2-eacbc0a60d6b admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:56.750301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ed862-3bc6-4553-b1f2-eacbc0a60d6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:56.753095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ed862-3bc6-4553-b1f2-eacbc0a60d6b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:56.780482 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1037/2085] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:56 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:56.784479 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1ffd3-4c62-4c7e-9c12-dac4576a0949 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:56.796801 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1ffd3-4c62-4c7e-9c12-dac4576a0949 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:56.801532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcea91d-9127-41ba-b28f-bb9119cc8d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:56.802458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcea91d-9127-41ba-b28f-bb9119cc8d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:56.802878 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcea91d-9127-41ba-b28f-bb9119cc8d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:56.803254 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcea91d-9127-41ba-b28f-bb9119cc8d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:56.804085 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcea91d-9127-41ba-b28f-bb9119cc8d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:56.805795 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcea91d-9127-41ba-b28f-bb9119cc8d37 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:56.817746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1049/2086] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:00:56.821769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ed862-3bc6-4553-b1f2-eacbc0a60d6b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:56.833403 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ed862-3bc6-4553-b1f2-eacbc0a60d6b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:56.839143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b7651-ac8b-4aa2-983d-0711ea7aa440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:56.839809 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b7651-ac8b-4aa2-983d-0711ea7aa440 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:56.840090 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b7651-ac8b-4aa2-983d-0711ea7aa440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:56.840368 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b7651-ac8b-4aa2-983d-0711ea7aa440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:56.841090 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b7651-ac8b-4aa2-983d-0711ea7aa440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:56.844060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b7651-ac8b-4aa2-983d-0711ea7aa440 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:56.845498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:56.859240 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1038/2087] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:56.863188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcea91d-9127-41ba-b28f-bb9119cc8d37 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:56.875692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcea91d-9127-41ba-b28f-bb9119cc8d37 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:56.880646 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c3e29-e9d8-4b29-a62b-20e7602fef66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:56.881532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c3e29-e9d8-4b29-a62b-20e7602fef66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:56.881812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c3e29-e9d8-4b29-a62b-20e7602fef66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:56.883129 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c3e29-e9d8-4b29-a62b-20e7602fef66 admin admin] PATH_INFO: `/v3/projects/7f0e8b3eb8b249acafcaa24567f9e985/users/1abaef7f35694822866f3b0bcb54871e/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:56.896896 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:56.901051 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c3e29-e9d8-4b29-a62b-20e7602fef66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f0e8b3eb8b249acafcaa24567f9e985, user_id=1abaef7f35694822866f3b0bcb54871e, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:56.904978 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c3e29-e9d8-4b29-a62b-20e7602fef66 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:56.939374 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1050/2088] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:56 2022] PUT /identity/v3/projects/7f0e8b3eb8b249acafcaa24567f9e985/users/1abaef7f35694822866f3b0bcb54871e/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:00:56.941075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae04719-1eb8-4d91-9d66-8c0a570ab20c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:56.941377 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae04719-1eb8-4d91-9d66-8c0a570ab20c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:56.941659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae04719-1eb8-4d91-9d66-8c0a570ab20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:56.983849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-6ae04719-1eb8-4d91-9d66-8c0a570ab20c None None] MFA Rules not processed for user `1abaef7f35694822866f3b0bcb54871e`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:56.985312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6ae04719-1eb8-4d91-9d66-8c0a570ab20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:57.062675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1039/2089] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:57.067168 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c3e29-e9d8-4b29-a62b-20e7602fef66 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:57.143272 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c3e29-e9d8-4b29-a62b-20e7602fef66 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:57.148781 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd62c6f-449c-426e-9868-4ca4da8881e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:57.149362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd62c6f-449c-426e-9868-4ca4da8881e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:57.149784 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd62c6f-449c-426e-9868-4ca4da8881e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:57.150191 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd62c6f-449c-426e-9868-4ca4da8881e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:57.151394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd62c6f-449c-426e-9868-4ca4da8881e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:57.154279 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd62c6f-449c-426e-9868-4ca4da8881e2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:57.190224 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1051/2090] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:57 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:57.193966 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae04719-1eb8-4d91-9d66-8c0a570ab20c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:57.205975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae04719-1eb8-4d91-9d66-8c0a570ab20c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:57.320661 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c6d93-dbfc-4ac9-923b-6ca0b7e1586c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:57.321398 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c6d93-dbfc-4ac9-923b-6ca0b7e1586c admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:57.321538 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c6d93-dbfc-4ac9-923b-6ca0b7e1586c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:57.321704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c6d93-dbfc-4ac9-923b-6ca0b7e1586c admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:57.322419 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c6d93-dbfc-4ac9-923b-6ca0b7e1586c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:57.325114 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c6d93-dbfc-4ac9-923b-6ca0b7e1586c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:57.359745 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1040/2091] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:57 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:57.364177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd62c6f-449c-426e-9868-4ca4da8881e2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:57.379740 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd62c6f-449c-426e-9868-4ca4da8881e2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:57.385546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dac71c-92f1-4433-a0f7-559354abbb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:57.386559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dac71c-92f1-4433-a0f7-559354abbb35 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:57.387055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dac71c-92f1-4433-a0f7-559354abbb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:57.387514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dac71c-92f1-4433-a0f7-559354abbb35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:57.389595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dac71c-92f1-4433-a0f7-559354abbb35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:57.393176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dac71c-92f1-4433-a0f7-559354abbb35 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:57.406890 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1052/2092] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:57.410820 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c6d93-dbfc-4ac9-923b-6ca0b7e1586c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:57.421707 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c6d93-dbfc-4ac9-923b-6ca0b7e1586c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:57.427176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92073226-c445-4d92-8452-14c74ea19319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:57.428446 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92073226-c445-4d92-8452-14c74ea19319 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:57.428824 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92073226-c445-4d92-8452-14c74ea19319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:57.429204 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92073226-c445-4d92-8452-14c74ea19319 admin admin] PATH_INFO: `/v3/projects/6ea242e7b6e3431ca5fe7ccff1108ffe/users/009c69b3a8c94936b282c52850697318/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:57.446302 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92073226-c445-4d92-8452-14c74ea19319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ea242e7b6e3431ca5fe7ccff1108ffe, user_id=009c69b3a8c94936b282c52850697318, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:57.464762 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92073226-c445-4d92-8452-14c74ea19319 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:57.491677 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1041/2093] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:57 2022] PUT /identity/v3/projects/6ea242e7b6e3431ca5fe7ccff1108ffe/users/009c69b3a8c94936b282c52850697318/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:00:57.496127 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dac71c-92f1-4433-a0f7-559354abbb35 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:57.507537 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dac71c-92f1-4433-a0f7-559354abbb35 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:57.583904 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887e6a7-332c-4321-8d13-5ed74e6b340d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:57.584599 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887e6a7-332c-4321-8d13-5ed74e6b340d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:57.584882 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887e6a7-332c-4321-8d13-5ed74e6b340d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:57.585171 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887e6a7-332c-4321-8d13-5ed74e6b340d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:57.586163 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887e6a7-332c-4321-8d13-5ed74e6b340d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:57.587806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887e6a7-332c-4321-8d13-5ed74e6b340d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:57.603367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1053/2094] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:57.606563 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92073226-c445-4d92-8452-14c74ea19319 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:57.623608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92073226-c445-4d92-8452-14c74ea19319 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:57.629256 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cebfa-11a6-4337-a471-b4f7fe53face admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:57.630190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cebfa-11a6-4337-a471-b4f7fe53face admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:57.630474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cebfa-11a6-4337-a471-b4f7fe53face admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:57.630798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cebfa-11a6-4337-a471-b4f7fe53face admin admin] PATH_INFO: `/v3/projects/6ea242e7b6e3431ca5fe7ccff1108ffe/users/009c69b3a8c94936b282c52850697318/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:57.632849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041cebfa-11a6-4337-a471-b4f7fe53face admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ea242e7b6e3431ca5fe7ccff1108ffe, user_id=009c69b3a8c94936b282c52850697318, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:57.635670 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041cebfa-11a6-4337-a471-b4f7fe53face admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:57.647223 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1042/2095] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:57 2022] PUT /identity/v3/projects/6ea242e7b6e3431ca5fe7ccff1108ffe/users/009c69b3a8c94936b282c52850697318/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:57.658778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472b379-7a8d-4d20-b443-0e040c5cb11f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:57.659166 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472b379-7a8d-4d20-b443-0e040c5cb11f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:57.659462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472b379-7a8d-4d20-b443-0e040c5cb11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:57.686968 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8472b379-7a8d-4d20-b443-0e040c5cb11f None None] MFA Rules not processed for user `009c69b3a8c94936b282c52850697318`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:57.688431 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8472b379-7a8d-4d20-b443-0e040c5cb11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:57.738076 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:57.759862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1054/2096] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:00:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:57.773122 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cebfa-11a6-4337-a471-b4f7fe53face admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:57.777197 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-041cebfa-11a6-4337-a471-b4f7fe53face admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:57.845455 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:57.867714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cebfa-11a6-4337-a471-b4f7fe53face admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:57.873575 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d3383-6506-4b43-9278-8ff8bd6060ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:57.874500 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d3383-6506-4b43-9278-8ff8bd6060ef service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:57.875067 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d3383-6506-4b43-9278-8ff8bd6060ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:57.875577 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d3383-6506-4b43-9278-8ff8bd6060ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:57.955386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d3383-6506-4b43-9278-8ff8bd6060ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:57.957636 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d3383-6506-4b43-9278-8ff8bd6060ef service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:57.978962 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1043/2097] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:00:59.703094 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472b379-7a8d-4d20-b443-0e040c5cb11f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:59.704761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8472b379-7a8d-4d20-b443-0e040c5cb11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:59.723211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472b379-7a8d-4d20-b443-0e040c5cb11f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:59.727894 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401821c-359b-4bb7-93f5-b430f17a312e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:59.728555 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401821c-359b-4bb7-93f5-b430f17a312e service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:59.728854 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401821c-359b-4bb7-93f5-b430f17a312e service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:59.729785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401821c-359b-4bb7-93f5-b430f17a312e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:59.939185 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d401821c-359b-4bb7-93f5-b430f17a312e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:59.941887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d401821c-359b-4bb7-93f5-b430f17a312e service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:59.959148 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1055/2098] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:00.899164 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d3383-6506-4b43-9278-8ff8bd6060ef service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:00.912176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d3383-6506-4b43-9278-8ff8bd6060ef service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:00.917980 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b91da-14a8-4c45-91dd-960870d2555e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:00.919202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b91da-14a8-4c45-91dd-960870d2555e service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:00.919677 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b91da-14a8-4c45-91dd-960870d2555e service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:00.920297 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b91da-14a8-4c45-91dd-960870d2555e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:00.934855 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b91da-14a8-4c45-91dd-960870d2555e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:00.936831 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b91da-14a8-4c45-91dd-960870d2555e service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:00.952579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1044/2099] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:01.137050 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b950b4-fd41-4640-850a-a02a29696a93 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:01.137567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b950b4-fd41-4640-850a-a02a29696a93 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:01.137676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b950b4-fd41-4640-850a-a02a29696a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:01.176629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-13b950b4-fd41-4640-850a-a02a29696a93 None None] MFA Rules not processed for user `047e841cefdd4975947b2c154412953b`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:01.179023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-13b950b4-fd41-4640-850a-a02a29696a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:01.319704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1056/2100] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:01.320792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b91da-14a8-4c45-91dd-960870d2555e service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:01.322727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-333b91da-14a8-4c45-91dd-960870d2555e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:01.469598 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b91da-14a8-4c45-91dd-960870d2555e service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:01.476347 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51566b-640f-4768-9b4a-ef22e0a3a263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:01.477195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51566b-640f-4768-9b4a-ef22e0a3a263 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:01.477603 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51566b-640f-4768-9b4a-ef22e0a3a263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:01.478015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51566b-640f-4768-9b4a-ef22e0a3a263 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:01.479037 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51566b-640f-4768-9b4a-ef22e0a3a263 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:01.482566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51566b-640f-4768-9b4a-ef22e0a3a263 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:01.550470 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1045/2101] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:01:01 2022] POST /identity/v3/projects => generated 383 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:01.552096 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b950b4-fd41-4640-850a-a02a29696a93 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:01.569490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b950b4-fd41-4640-850a-a02a29696a93 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:01.672223 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fc1b1-fa32-4650-aeab-f1b5019d0623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:01.673525 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fc1b1-fa32-4650-aeab-f1b5019d0623 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:01.673770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fc1b1-fa32-4650-aeab-f1b5019d0623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:01.674004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fc1b1-fa32-4650-aeab-f1b5019d0623 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:01.674769 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fc1b1-fa32-4650-aeab-f1b5019d0623 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:01.676864 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fc1b1-fa32-4650-aeab-f1b5019d0623 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:01.706199 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1057/2102] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:01:01 2022] POST /identity/v3/users => generated 298 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:01.710913 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51566b-640f-4768-9b4a-ef22e0a3a263 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:01.728039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51566b-640f-4768-9b4a-ef22e0a3a263 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:01.735126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb35a6e-5d65-4482-853c-2bdef1fde3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:01.736055 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb35a6e-5d65-4482-853c-2bdef1fde3e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:01.736408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb35a6e-5d65-4482-853c-2bdef1fde3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:01.736749 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb35a6e-5d65-4482-853c-2bdef1fde3e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:01.737545 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb35a6e-5d65-4482-853c-2bdef1fde3e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:01.739413 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb35a6e-5d65-4482-853c-2bdef1fde3e3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:01.754610 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1046/2103] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:01.756301 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fc1b1-fa32-4650-aeab-f1b5019d0623 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:01.763098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:01.767875 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fc1b1-fa32-4650-aeab-f1b5019d0623 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:01.774362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07e0d8-6aa8-41f2-99c2-e898d1088290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:01.775461 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07e0d8-6aa8-41f2-99c2-e898d1088290 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:01.775746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07e0d8-6aa8-41f2-99c2-e898d1088290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:01.776316 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07e0d8-6aa8-41f2-99c2-e898d1088290 admin admin] PATH_INFO: `/v3/projects/4eafa3e811a6497faa943075af273826/users/0fd50fda92344a1d81c3eefd8e2f2508/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:01.796253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07e0d8-6aa8-41f2-99c2-e898d1088290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eafa3e811a6497faa943075af273826, user_id=0fd50fda92344a1d81c3eefd8e2f2508, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:01.799111 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07e0d8-6aa8-41f2-99c2-e898d1088290 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:01.814208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1058/2104] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:01 2022] PUT /identity/v3/projects/4eafa3e811a6497faa943075af273826/users/0fd50fda92344a1d81c3eefd8e2f2508/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:01.818185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb35a6e-5d65-4482-853c-2bdef1fde3e3 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:01.829057 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb35a6e-5d65-4482-853c-2bdef1fde3e3 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:01.885635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:01.900717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f229354-cda7-4152-b1ed-2a6038b8eee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:01.901716 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f229354-cda7-4152-b1ed-2a6038b8eee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:01.902079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f229354-cda7-4152-b1ed-2a6038b8eee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:01.902255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f229354-cda7-4152-b1ed-2a6038b8eee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:01.903031 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f229354-cda7-4152-b1ed-2a6038b8eee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:01.906387 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f229354-cda7-4152-b1ed-2a6038b8eee7 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:01.918222 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1047/2105] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:01.924231 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07e0d8-6aa8-41f2-99c2-e898d1088290 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:01.936652 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07e0d8-6aa8-41f2-99c2-e898d1088290 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:01.941647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153321ca-7809-4e81-974d-583801de28d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:01.942550 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153321ca-7809-4e81-974d-583801de28d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:01.942870 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153321ca-7809-4e81-974d-583801de28d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:01.948553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153321ca-7809-4e81-974d-583801de28d8 admin admin] PATH_INFO: `/v3/projects/4eafa3e811a6497faa943075af273826/users/0fd50fda92344a1d81c3eefd8e2f2508/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:01.952703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153321ca-7809-4e81-974d-583801de28d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eafa3e811a6497faa943075af273826, user_id=0fd50fda92344a1d81c3eefd8e2f2508, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:01.956428 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153321ca-7809-4e81-974d-583801de28d8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:01.974723 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1059/2106] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:01 2022] PUT /identity/v3/projects/4eafa3e811a6497faa943075af273826/users/0fd50fda92344a1d81c3eefd8e2f2508/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:05.193663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f229354-cda7-4152-b1ed-2a6038b8eee7 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:05.196443 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5f229354-cda7-4152-b1ed-2a6038b8eee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:05.219923 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f229354-cda7-4152-b1ed-2a6038b8eee7 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:05.253793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153321ca-7809-4e81-974d-583801de28d8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:05.387093 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153321ca-7809-4e81-974d-583801de28d8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:05.388979 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065b452-269f-4fc8-8673-f367b1c98745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:05.390244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065b452-269f-4fc8-8673-f367b1c98745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:05.392459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065b452-269f-4fc8-8673-f367b1c98745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:05.392891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065b452-269f-4fc8-8673-f367b1c98745 admin admin] PATH_INFO: `/v3/users/8dd2c077b8d3430f805e5dd4bc196fcd` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:05.395478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065b452-269f-4fc8-8673-f367b1c98745 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dd2c077b8d3430f805e5dd4bc196fcd)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:05.401504 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065b452-269f-4fc8-8673-f367b1c98745 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:05.407646 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317c677-8048-48f3-adb2-aeb868bdd472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:05.408863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317c677-8048-48f3-adb2-aeb868bdd472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:05.409818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317c677-8048-48f3-adb2-aeb868bdd472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:05.410385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317c677-8048-48f3-adb2-aeb868bdd472 admin admin] PATH_INFO: `/v3/users/1001192119c04a1fb09938c41fd0d4f2` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:05.412256 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317c677-8048-48f3-adb2-aeb868bdd472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1001192119c04a1fb09938c41fd0d4f2)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:05.414690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317c677-8048-48f3-adb2-aeb868bdd472 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:05.495589 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4317c677-8048-48f3-adb2-aeb868bdd472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1001192119c04a1fb09938c41fd0d4f2'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:05.496338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4317c677-8048-48f3-adb2-aeb868bdd472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1001192119c04a1fb09938c41fd0d4f2'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:05.496534 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b065b452-269f-4fc8-8673-f367b1c98745 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dd2c077b8d3430f805e5dd4bc196fcd'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:05.497007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4317c677-8048-48f3-adb2-aeb868bdd472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1001192119c04a1fb09938c41fd0d4f2'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:05.497521 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b065b452-269f-4fc8-8673-f367b1c98745 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dd2c077b8d3430f805e5dd4bc196fcd'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:05.508525 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4317c677-8048-48f3-adb2-aeb868bdd472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1001192119c04a1fb09938c41fd0d4f2'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:05.518816 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4317c677-8048-48f3-adb2-aeb868bdd472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1001192119c04a1fb09938c41fd0d4f2'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:05.522223 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b065b452-269f-4fc8-8673-f367b1c98745 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dd2c077b8d3430f805e5dd4bc196fcd'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:05.542905 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1060/2107] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:01:05 2022] DELETE /identity/v3/users/1001192119c04a1fb09938c41fd0d4f2 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:05.549759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73e003-68ff-4537-80cf-222b58ec721f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:05.550089 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73e003-68ff-4537-80cf-222b58ec721f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:05.550391 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73e003-68ff-4537-80cf-222b58ec721f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:05.555773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b065b452-269f-4fc8-8673-f367b1c98745 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dd2c077b8d3430f805e5dd4bc196fcd'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:05.558677 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b065b452-269f-4fc8-8673-f367b1c98745 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dd2c077b8d3430f805e5dd4bc196fcd'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:05.570492 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2b73e003-68ff-4537-80cf-222b58ec721f None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:05.572637 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2b73e003-68ff-4537-80cf-222b58ec721f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:05.594286 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1048/2108] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:01:05 2022] DELETE /identity/v3/users/8dd2c077b8d3430f805e5dd4bc196fcd => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:05.598833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ddf75-3902-47f6-8ec2-eed422bda433 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:05.599171 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ddf75-3902-47f6-8ec2-eed422bda433 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:05.599586 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ddf75-3902-47f6-8ec2-eed422bda433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:05.628624 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c49ddf75-3902-47f6-8ec2-eed422bda433 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:05.630659 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c49ddf75-3902-47f6-8ec2-eed422bda433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:05.635037 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:05.664966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1061/2109] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:05.679477 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73e003-68ff-4537-80cf-222b58ec721f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:05.681237 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2b73e003-68ff-4537-80cf-222b58ec721f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:05.782952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1049/2110] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:01:05.806515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ddf75-3902-47f6-8ec2-eed422bda433 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:05.821048 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:05.821823 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73e003-68ff-4537-80cf-222b58ec721f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:05.948549 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ddf75-3902-47f6-8ec2-eed422bda433 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:05.954216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67251ed0-7b32-4b8b-a80f-5226dbc205a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:05.960124 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67251ed0-7b32-4b8b-a80f-5226dbc205a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:05.960124 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67251ed0-7b32-4b8b-a80f-5226dbc205a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:05.960124 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67251ed0-7b32-4b8b-a80f-5226dbc205a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:05.961021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5846bc-91f9-467f-a4be-9a40edb01305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:05.961021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5846bc-91f9-467f-a4be-9a40edb01305 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:05.961021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5846bc-91f9-467f-a4be-9a40edb01305 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:05.961021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5846bc-91f9-467f-a4be-9a40edb01305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:05.976539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67251ed0-7b32-4b8b-a80f-5226dbc205a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:05.979884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67251ed0-7b32-4b8b-a80f-5226dbc205a3 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:05.985079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5846bc-91f9-467f-a4be-9a40edb01305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:05.986556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5846bc-91f9-467f-a4be-9a40edb01305 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:06.006980 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1050/2111] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:06.010538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1062/2112] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:06.912303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5846bc-91f9-467f-a4be-9a40edb01305 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:06.914272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3a5846bc-91f9-467f-a4be-9a40edb01305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:06.948326 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5846bc-91f9-467f-a4be-9a40edb01305 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:06.968035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27a9a7-2589-46db-ad48-7c89a31ac466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:06.973234 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27a9a7-2589-46db-ad48-7c89a31ac466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:06.973649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27a9a7-2589-46db-ad48-7c89a31ac466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:06.974156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27a9a7-2589-46db-ad48-7c89a31ac466 admin admin] PATH_INFO: `/v3/projects/a3bd32bf0f224fb8be1bf288e72d6614` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:06.976856 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27a9a7-2589-46db-ad48-7c89a31ac466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3bd32bf0f224fb8be1bf288e72d6614)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:07.003907 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27a9a7-2589-46db-ad48-7c89a31ac466 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:07.132956 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-0b27a9a7-2589-46db-ad48-7c89a31ac466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3bd32bf0f224fb8be1bf288e72d6614', 'request_id': 'req-0b27a9a7-2589-46db-ad48-7c89a31ac466'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:07.147656 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-0b27a9a7-2589-46db-ad48-7c89a31ac466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3bd32bf0f224fb8be1bf288e72d6614', 'request_id': 'req-0b27a9a7-2589-46db-ad48-7c89a31ac466'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:07.171719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-0b27a9a7-2589-46db-ad48-7c89a31ac466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3bd32bf0f224fb8be1bf288e72d6614', 'request_id': 'req-0b27a9a7-2589-46db-ad48-7c89a31ac466'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:07.191125 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-0b27a9a7-2589-46db-ad48-7c89a31ac466 admin admin] The token cache is being invalidate because project a3bd32bf0f224fb8be1bf288e72d6614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:01:07.191526 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-0b27a9a7-2589-46db-ad48-7c89a31ac466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:07.192465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-0b27a9a7-2589-46db-ad48-7c89a31ac466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:07.194710 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1051/2113] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:01:06 2022] DELETE /identity/v3/projects/a3bd32bf0f224fb8be1bf288e72d6614 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:07.198522 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67251ed0-7b32-4b8b-a80f-5226dbc205a3 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:07.211532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67251ed0-7b32-4b8b-a80f-5226dbc205a3 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:07.453868 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27a9a7-2589-46db-ad48-7c89a31ac466 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:07.457258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0b27a9a7-2589-46db-ad48-7c89a31ac466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:07.464549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77436522-b970-491e-ab99-7381235d25d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:07.466678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77436522-b970-491e-ab99-7381235d25d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:07.467349 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77436522-b970-491e-ab99-7381235d25d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:07.467670 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77436522-b970-491e-ab99-7381235d25d9 admin admin] PATH_INFO: `/v3/users/1abaef7f35694822866f3b0bcb54871e` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:07.482118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77436522-b970-491e-ab99-7381235d25d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1abaef7f35694822866f3b0bcb54871e)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:07.485539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77436522-b970-491e-ab99-7381235d25d9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:07.514638 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27a9a7-2589-46db-ad48-7c89a31ac466 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:07.555867 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d84d97-2938-4103-874d-85d40cc198b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:07.557191 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d84d97-2938-4103-874d-85d40cc198b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:07.557690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d84d97-2938-4103-874d-85d40cc198b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:07.558434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d84d97-2938-4103-874d-85d40cc198b8 admin admin] PATH_INFO: `/v3/projects/7fb98b4301b343f392b4a466ad5c7b40` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:07.576684 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d84d97-2938-4103-874d-85d40cc198b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fb98b4301b343f392b4a466ad5c7b40)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:07.584023 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d84d97-2938-4103-874d-85d40cc198b8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:07.699462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-77436522-b970-491e-ab99-7381235d25d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1abaef7f35694822866f3b0bcb54871e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:07.701192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-77436522-b970-491e-ab99-7381235d25d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1abaef7f35694822866f3b0bcb54871e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:07.702398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-77436522-b970-491e-ab99-7381235d25d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1abaef7f35694822866f3b0bcb54871e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:07.704722 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:07.716943 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-77436522-b970-491e-ab99-7381235d25d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1abaef7f35694822866f3b0bcb54871e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:07.734691 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-77436522-b970-491e-ab99-7381235d25d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1abaef7f35694822866f3b0bcb54871e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:07.822641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1063/2114] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:01:07 2022] DELETE /identity/v3/users/1abaef7f35694822866f3b0bcb54871e => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:07.853477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d3d84d97-2938-4103-874d-85d40cc198b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fb98b4301b343f392b4a466ad5c7b40', 'request_id': 'req-d3d84d97-2938-4103-874d-85d40cc198b8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:07.866400 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d3d84d97-2938-4103-874d-85d40cc198b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fb98b4301b343f392b4a466ad5c7b40', 'request_id': 'req-d3d84d97-2938-4103-874d-85d40cc198b8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:07.888549 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d3d84d97-2938-4103-874d-85d40cc198b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fb98b4301b343f392b4a466ad5c7b40', 'request_id': 'req-d3d84d97-2938-4103-874d-85d40cc198b8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:07.926788 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d3d84d97-2938-4103-874d-85d40cc198b8 admin admin] The token cache is being invalidate because project 7fb98b4301b343f392b4a466ad5c7b40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:01:07.927531 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d3d84d97-2938-4103-874d-85d40cc198b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:07.928140 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d3d84d97-2938-4103-874d-85d40cc198b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:07.933120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1052/2115] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:01:07 2022] DELETE /identity/v3/projects/7fb98b4301b343f392b4a466ad5c7b40 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:01:07.939822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a311b36-4c58-47b1-884e-39f54886cb1d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:07.940155 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a311b36-4c58-47b1-884e-39f54886cb1d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:07.940524 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a311b36-4c58-47b1-884e-39f54886cb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:08.011317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5a311b36-4c58-47b1-884e-39f54886cb1d None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:08.017297 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5a311b36-4c58-47b1-884e-39f54886cb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:08.213586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1064/2116] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:08.221412 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a94f24-5dc4-4347-877e-954ff677bce8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:08.221958 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a94f24-5dc4-4347-877e-954ff677bce8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:08.222429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a94f24-5dc4-4347-877e-954ff677bce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:08.245588 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-73a94f24-5dc4-4347-877e-954ff677bce8 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:08.248258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-73a94f24-5dc4-4347-877e-954ff677bce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:08.257416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1053/2117] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:08.261445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a311b36-4c58-47b1-884e-39f54886cb1d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:08.272914 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a311b36-4c58-47b1-884e-39f54886cb1d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:08.284582 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb10689-94e3-43d0-b38e-6addf39e041c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:08.286167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb10689-94e3-43d0-b38e-6addf39e041c admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:08.286698 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb10689-94e3-43d0-b38e-6addf39e041c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:08.287352 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb10689-94e3-43d0-b38e-6addf39e041c admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:08.288273 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb10689-94e3-43d0-b38e-6addf39e041c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:01:08.288757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb10689-94e3-43d0-b38e-6addf39e041c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:08.291501 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb10689-94e3-43d0-b38e-6addf39e041c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:08.313081 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1065/2118] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:01:08.317694 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a94f24-5dc4-4347-877e-954ff677bce8 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:08.330384 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a94f24-5dc4-4347-877e-954ff677bce8 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:08.336643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9137815-242c-40cf-9ba5-0735781220b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:08.339875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9137815-242c-40cf-9ba5-0735781220b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:08.340668 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9137815-242c-40cf-9ba5-0735781220b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:08.341701 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9137815-242c-40cf-9ba5-0735781220b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:08.344695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9137815-242c-40cf-9ba5-0735781220b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:08.347314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9137815-242c-40cf-9ba5-0735781220b4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:08.372264 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:08.439882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1054/2119] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:01:08 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:01:08.441245 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb10689-94e3-43d0-b38e-6addf39e041c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:08.470608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb10689-94e3-43d0-b38e-6addf39e041c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:08.711305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecedff9-249c-427d-b724-6f93a2fc383d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:08.712038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecedff9-249c-427d-b724-6f93a2fc383d admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:08.712328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecedff9-249c-427d-b724-6f93a2fc383d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:08.712610 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecedff9-249c-427d-b724-6f93a2fc383d admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:08.713335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecedff9-249c-427d-b724-6f93a2fc383d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:08.718564 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecedff9-249c-427d-b724-6f93a2fc383d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:08.779267 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1066/2120] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:01:08 2022] POST /identity/v3/users => generated 283 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:08.785158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9137815-242c-40cf-9ba5-0735781220b4 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:08.811138 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9137815-242c-40cf-9ba5-0735781220b4 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:08.829557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9aa48-b292-4405-8203-502342eaad13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:08.830256 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9aa48-b292-4405-8203-502342eaad13 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:08.830535 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9aa48-b292-4405-8203-502342eaad13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:08.830859 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9aa48-b292-4405-8203-502342eaad13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:08.831595 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9aa48-b292-4405-8203-502342eaad13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:08.835288 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9aa48-b292-4405-8203-502342eaad13 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:08.864539 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1055/2121] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:08.868434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecedff9-249c-427d-b724-6f93a2fc383d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:08.908030 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecedff9-249c-427d-b724-6f93a2fc383d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:08.915458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac355a-f7c5-4bbf-8a4f-35dd939ba3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:08.916694 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac355a-f7c5-4bbf-8a4f-35dd939ba3fb admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:08.917262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac355a-f7c5-4bbf-8a4f-35dd939ba3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:08.917888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac355a-f7c5-4bbf-8a4f-35dd939ba3fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:08.919495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac355a-f7c5-4bbf-8a4f-35dd939ba3fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:08.922484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac355a-f7c5-4bbf-8a4f-35dd939ba3fb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:09.019000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1067/2122] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:01:09.023539 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9aa48-b292-4405-8203-502342eaad13 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:09.040879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9aa48-b292-4405-8203-502342eaad13 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:09.085640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48e107-222b-4d52-902c-0a76f67f79a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:09.087303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48e107-222b-4d52-902c-0a76f67f79a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:09.087760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48e107-222b-4d52-902c-0a76f67f79a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:09.088249 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48e107-222b-4d52-902c-0a76f67f79a1 admin admin] PATH_INFO: `/v3/projects/b585ac2789ae4143af8a40926b14101c/users/292561b2e6d6408c928c6302b63ad9c7/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:09.125452 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48e107-222b-4d52-902c-0a76f67f79a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b585ac2789ae4143af8a40926b14101c, user_id=292561b2e6d6408c928c6302b63ad9c7, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:09.132261 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48e107-222b-4d52-902c-0a76f67f79a1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:09.166463 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1056/2123] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:09 2022] PUT /identity/v3/projects/b585ac2789ae4143af8a40926b14101c/users/292561b2e6d6408c928c6302b63ad9c7/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:09.178737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca18721-62a5-4d04-8a3b-5d70faffd359 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:09.179201 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca18721-62a5-4d04-8a3b-5d70faffd359 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:09.179582 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca18721-62a5-4d04-8a3b-5d70faffd359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:09.258638 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-eca18721-62a5-4d04-8a3b-5d70faffd359 None None] MFA Rules not processed for user `292561b2e6d6408c928c6302b63ad9c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:09.260691 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-eca18721-62a5-4d04-8a3b-5d70faffd359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:09.285484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:09.425959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1068/2124] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:09.438862 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48e107-222b-4d52-902c-0a76f67f79a1 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:09.445675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6f48e107-222b-4d52-902c-0a76f67f79a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:09.562001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48e107-222b-4d52-902c-0a76f67f79a1 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:09.579132 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d21bf-6025-45d2-a343-2f474298c723 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:09.583876 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d21bf-6025-45d2-a343-2f474298c723 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:09.585795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d21bf-6025-45d2-a343-2f474298c723 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:09.586133 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d21bf-6025-45d2-a343-2f474298c723 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:09.618257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d21bf-6025-45d2-a343-2f474298c723 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:09.622743 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d21bf-6025-45d2-a343-2f474298c723 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:09.682529 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1057/2125] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:10.379393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7e8a6-fdf0-4a19-ab8f-389b60d465a5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:10.379857 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7e8a6-fdf0-4a19-ab8f-389b60d465a5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:10.379977 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7e8a6-fdf0-4a19-ab8f-389b60d465a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:10.452967 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d3e7e8a6-fdf0-4a19-ab8f-389b60d465a5 None None] MFA Rules not processed for user `0fd50fda92344a1d81c3eefd8e2f2508`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:10.452967 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d3e7e8a6-fdf0-4a19-ab8f-389b60d465a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:10.537739 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d21bf-6025-45d2-a343-2f474298c723 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:10.539829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-253d21bf-6025-45d2-a343-2f474298c723 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:10.624608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1069/2126] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:10.641585 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7e8a6-fdf0-4a19-ab8f-389b60d465a5 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:10.643184 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d3e7e8a6-fdf0-4a19-ab8f-389b60d465a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:10.721324 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d21bf-6025-45d2-a343-2f474298c723 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:10.734879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042200b9-c941-4b07-8c14-b75dc095d886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:10.736273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042200b9-c941-4b07-8c14-b75dc095d886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:10.736727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042200b9-c941-4b07-8c14-b75dc095d886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:10.737293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042200b9-c941-4b07-8c14-b75dc095d886 admin admin] PATH_INFO: `/v3/projects/7f0e8b3eb8b249acafcaa24567f9e985` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:10.746171 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042200b9-c941-4b07-8c14-b75dc095d886 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f0e8b3eb8b249acafcaa24567f9e985)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:10.748344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042200b9-c941-4b07-8c14-b75dc095d886 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:10.776707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7e8a6-fdf0-4a19-ab8f-389b60d465a5 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:10.787991 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616f020-369f-42c4-ace7-48ad8929058d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8563a25d16134d639a2a8849976fed9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:10.788679 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616f020-369f-42c4-ace7-48ad8929058d service glance] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:10.788972 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616f020-369f-42c4-ace7-48ad8929058d service glance] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:10.789261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616f020-369f-42c4-ace7-48ad8929058d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:10.793279 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e616f020-369f-42c4-ace7-48ad8929058d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:10.846997 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-042200b9-c941-4b07-8c14-b75dc095d886 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f0e8b3eb8b249acafcaa24567f9e985', 'request_id': 'req-042200b9-c941-4b07-8c14-b75dc095d886'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:10.876379 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-042200b9-c941-4b07-8c14-b75dc095d886 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f0e8b3eb8b249acafcaa24567f9e985', 'request_id': 'req-042200b9-c941-4b07-8c14-b75dc095d886'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:10.903715 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:10.921022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616f020-369f-42c4-ace7-48ad8929058d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:10.921312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-042200b9-c941-4b07-8c14-b75dc095d886 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f0e8b3eb8b249acafcaa24567f9e985', 'request_id': 'req-042200b9-c941-4b07-8c14-b75dc095d886'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:10.922899 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616f020-369f-42c4-ace7-48ad8929058d service glance] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:10.938819 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:10.948937 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1070/2127] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:01:10.963497 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-042200b9-c941-4b07-8c14-b75dc095d886 admin admin] The token cache is being invalidate because project 7f0e8b3eb8b249acafcaa24567f9e985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:01:10.965404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-042200b9-c941-4b07-8c14-b75dc095d886 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:10.966739 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-042200b9-c941-4b07-8c14-b75dc095d886 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:10.972284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1058/2128] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:01:10 2022] DELETE /identity/v3/projects/7f0e8b3eb8b249acafcaa24567f9e985 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:10.976159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616f020-369f-42c4-ace7-48ad8929058d service glance] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:10.992149 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616f020-369f-42c4-ace7-48ad8929058d service glance] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:11.017417 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f4fb0-6149-4775-8b06-e5107414e677 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:11.017891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f4fb0-6149-4775-8b06-e5107414e677 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:11.018325 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f4fb0-6149-4775-8b06-e5107414e677 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:11.019869 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1059/2129] 10.222.0.70 () {54 vars in 921 bytes} [Thu Jun 16 19:01:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:01:11.029921 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea30bf4-6e70-4e46-98d9-4a88a4c8d8d0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:11.030891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea30bf4-6e70-4e46-98d9-4a88a4c8d8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:11.031320 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea30bf4-6e70-4e46-98d9-4a88a4c8d8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:11.080952 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-6ea30bf4-6e70-4e46-98d9-4a88a4c8d8d0 None None] MFA Rules not processed for user `880d0299c38c4eb897c669a512e599be`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:11.082161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6ea30bf4-6e70-4e46-98d9-4a88a4c8d8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:11.232993 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d6a537-5c6b-4f6a-a886-2e064a325148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:11.235248 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6a537-5c6b-4f6a-a886-2e064a325148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:11.235537 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6a537-5c6b-4f6a-a886-2e064a325148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:11.235813 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6a537-5c6b-4f6a-a886-2e064a325148 admin admin] PATH_INFO: `/v3/users/009c69b3a8c94936b282c52850697318` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:11.237861 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d6a537-5c6b-4f6a-a886-2e064a325148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=009c69b3a8c94936b282c52850697318)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:11.240700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d6a537-5c6b-4f6a-a886-2e064a325148 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:11.283802 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1060/2130] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Jun 16 19:01:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:11.284680 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea30bf4-6e70-4e46-98d9-4a88a4c8d8d0 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:11.286042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6ea30bf4-6e70-4e46-98d9-4a88a4c8d8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:11.308516 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea30bf4-6e70-4e46-98d9-4a88a4c8d8d0 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:11.321531 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5afad-5b05-4bc6-b73b-22d9e240a2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:11.322291 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5afad-5b05-4bc6-b73b-22d9e240a2eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:11.322662 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5afad-5b05-4bc6-b73b-22d9e240a2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:11.322843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5afad-5b05-4bc6-b73b-22d9e240a2eb admin admin] PATH_INFO: `/v3/users/e8813b3f6f8c46519fb827cf521bb37b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:11.325080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5afad-5b05-4bc6-b73b-22d9e240a2eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8813b3f6f8c46519fb827cf521bb37b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:11.327653 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5afad-5b05-4bc6-b73b-22d9e240a2eb admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:11.333717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c7d6a537-5c6b-4f6a-a886-2e064a325148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '009c69b3a8c94936b282c52850697318'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:11.334379 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c7d6a537-5c6b-4f6a-a886-2e064a325148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '009c69b3a8c94936b282c52850697318'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:11.335033 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c7d6a537-5c6b-4f6a-a886-2e064a325148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '009c69b3a8c94936b282c52850697318'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:11.355160 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c7d6a537-5c6b-4f6a-a886-2e064a325148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '009c69b3a8c94936b282c52850697318'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:11.377447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c7d6a537-5c6b-4f6a-a886-2e064a325148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '009c69b3a8c94936b282c52850697318'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:11.409143 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1071/2131] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:01:10 2022] DELETE /identity/v3/users/009c69b3a8c94936b282c52850697318 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:11.411898 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea37c38-c59b-42df-961d-83b3e017e76d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:11.412351 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea37c38-c59b-42df-961d-83b3e017e76d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:11.412947 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea37c38-c59b-42df-961d-83b3e017e76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:11.432011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f7a5afad-5b05-4bc6-b73b-22d9e240a2eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8813b3f6f8c46519fb827cf521bb37b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:11.433278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f7a5afad-5b05-4bc6-b73b-22d9e240a2eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8813b3f6f8c46519fb827cf521bb37b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:11.437361 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-dea37c38-c59b-42df-961d-83b3e017e76d None None] MFA Rules not processed for user `02c826fc0baa480393df4c884c9e35dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:11.439277 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-dea37c38-c59b-42df-961d-83b3e017e76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:11.445584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f7a5afad-5b05-4bc6-b73b-22d9e240a2eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8813b3f6f8c46519fb827cf521bb37b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:11.475599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f7a5afad-5b05-4bc6-b73b-22d9e240a2eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8813b3f6f8c46519fb827cf521bb37b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:11.476375 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f7a5afad-5b05-4bc6-b73b-22d9e240a2eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8813b3f6f8c46519fb827cf521bb37b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:11.496579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1061/2132] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:01:11 2022] DELETE /identity/v3/users/e8813b3f6f8c46519fb827cf521bb37b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:11.498514 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5afad-5b05-4bc6-b73b-22d9e240a2eb admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:11.500910 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f7a5afad-5b05-4bc6-b73b-22d9e240a2eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:11.588054 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1072/2133] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Jun 16 19:01:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:11.657671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea37c38-c59b-42df-961d-83b3e017e76d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:11.659267 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-dea37c38-c59b-42df-961d-83b3e017e76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:11.680806 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:11.723057 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5afad-5b05-4bc6-b73b-22d9e240a2eb admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:11.729429 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:11.745298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1a825-4d98-4fb2-9e9f-fd6ca2b6572d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '880d0299c38c4eb897c669a512e599be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:11.748712 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1a825-4d98-4fb2-9e9f-fd6ca2b6572d service placement] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:11.748845 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1a825-4d98-4fb2-9e9f-fd6ca2b6572d service placement] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:11.749218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1a825-4d98-4fb2-9e9f-fd6ca2b6572d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:11.760934 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-06d1a825-4d98-4fb2-9e9f-fd6ca2b6572d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:11.797576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1a825-4d98-4fb2-9e9f-fd6ca2b6572d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:11.800435 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1a825-4d98-4fb2-9e9f-fd6ca2b6572d service placement] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:11.818613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1062/2134] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:11.842829 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea37c38-c59b-42df-961d-83b3e017e76d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:12.004779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335025b5-4d4f-4f8a-b16a-d67926e43a00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b22fe0ab0354385924923d0d443acae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:12.005628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335025b5-4d4f-4f8a-b16a-d67926e43a00 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:12.005916 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335025b5-4d4f-4f8a-b16a-d67926e43a00 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:12.006190 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335025b5-4d4f-4f8a-b16a-d67926e43a00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:12.099645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335025b5-4d4f-4f8a-b16a-d67926e43a00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:12.101759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335025b5-4d4f-4f8a-b16a-d67926e43a00 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:12.129505 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1073/2135] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Jun 16 19:01:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 19:01:12.245838 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1a825-4d98-4fb2-9e9f-fd6ca2b6572d service placement] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:12.248948 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-06d1a825-4d98-4fb2-9e9f-fd6ca2b6572d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:12.490100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1a825-4d98-4fb2-9e9f-fd6ca2b6572d service placement] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:12.500277 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b482287-8d6f-443f-b9c6-3eda4b0dc860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:12.503378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b482287-8d6f-443f-b9c6-3eda4b0dc860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:12.503604 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b482287-8d6f-443f-b9c6-3eda4b0dc860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:12.504359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b482287-8d6f-443f-b9c6-3eda4b0dc860 admin admin] PATH_INFO: `/v3/projects/7de0652519d3447b8d0e4d948610071f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:12.509130 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b482287-8d6f-443f-b9c6-3eda4b0dc860 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7de0652519d3447b8d0e4d948610071f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:12.513754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b482287-8d6f-443f-b9c6-3eda4b0dc860 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:12.607416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7b482287-8d6f-443f-b9c6-3eda4b0dc860 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7de0652519d3447b8d0e4d948610071f', 'request_id': 'req-7b482287-8d6f-443f-b9c6-3eda4b0dc860'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:12.619218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7b482287-8d6f-443f-b9c6-3eda4b0dc860 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7de0652519d3447b8d0e4d948610071f', 'request_id': 'req-7b482287-8d6f-443f-b9c6-3eda4b0dc860'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:12.659558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7b482287-8d6f-443f-b9c6-3eda4b0dc860 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7de0652519d3447b8d0e4d948610071f', 'request_id': 'req-7b482287-8d6f-443f-b9c6-3eda4b0dc860'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:12.698750 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7b482287-8d6f-443f-b9c6-3eda4b0dc860 admin admin] The token cache is being invalidate because project 7de0652519d3447b8d0e4d948610071f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:01:12.699563 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7b482287-8d6f-443f-b9c6-3eda4b0dc860 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:12.702043 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7b482287-8d6f-443f-b9c6-3eda4b0dc860 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:12.705286 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1063/2136] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:01:12 2022] DELETE /identity/v3/projects/7de0652519d3447b8d0e4d948610071f => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:12.709909 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9140e-1a93-4b63-8e99-2de10e19216f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:12.710529 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9140e-1a93-4b63-8e99-2de10e19216f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:12.710697 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9140e-1a93-4b63-8e99-2de10e19216f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:12.741463 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-76f9140e-1a93-4b63-8e99-2de10e19216f None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:12.742476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-76f9140e-1a93-4b63-8e99-2de10e19216f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:12.844080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1074/2137] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:12.852376 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211cc7a-d4a0-4fc5-8646-349e827d6ca8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:12.852809 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211cc7a-d4a0-4fc5-8646-349e827d6ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:12.853643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211cc7a-d4a0-4fc5-8646-349e827d6ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:12.887063 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2211cc7a-d4a0-4fc5-8646-349e827d6ca8 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:12.890062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2211cc7a-d4a0-4fc5-8646-349e827d6ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:12.916477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1064/2138] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:01:12.921453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9140e-1a93-4b63-8e99-2de10e19216f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:12.936418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9140e-1a93-4b63-8e99-2de10e19216f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:12.952032 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acf5be-2588-4d3a-b51a-85cab0cd49ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:12.952715 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acf5be-2588-4d3a-b51a-85cab0cd49ac admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:12.953005 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acf5be-2588-4d3a-b51a-85cab0cd49ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:12.953284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acf5be-2588-4d3a-b51a-85cab0cd49ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:12.955050 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04acf5be-2588-4d3a-b51a-85cab0cd49ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:01:12.955050 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04acf5be-2588-4d3a-b51a-85cab0cd49ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:12.960393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04acf5be-2588-4d3a-b51a-85cab0cd49ac admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:12.966854 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:12.995075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1075/2139] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:01:12.999930 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2211cc7a-d4a0-4fc5-8646-349e827d6ca8 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:13.024217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2211cc7a-d4a0-4fc5-8646-349e827d6ca8 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:13.056948 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b1282-3040-473d-bf90-971879d6d343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:13.058182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b1282-3040-473d-bf90-971879d6d343 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:13.058740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b1282-3040-473d-bf90-971879d6d343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:13.059257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b1282-3040-473d-bf90-971879d6d343 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:13.072084 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b1282-3040-473d-bf90-971879d6d343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:13.080449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acf5be-2588-4d3a-b51a-85cab0cd49ac admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:13.083506 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-04acf5be-2588-4d3a-b51a-85cab0cd49ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:13.086413 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b1282-3040-473d-bf90-971879d6d343 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:13.111710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acf5be-2588-4d3a-b51a-85cab0cd49ac admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:13.125037 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40a6ce-8356-4579-85b6-33a1b0b555f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:13.126677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40a6ce-8356-4579-85b6-33a1b0b555f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:13.126677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40a6ce-8356-4579-85b6-33a1b0b555f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:13.126677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40a6ce-8356-4579-85b6-33a1b0b555f0 admin admin] PATH_INFO: `/v3/projects/6ea242e7b6e3431ca5fe7ccff1108ffe` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:13.129964 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40a6ce-8356-4579-85b6-33a1b0b555f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ea242e7b6e3431ca5fe7ccff1108ffe)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:13.132635 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40a6ce-8356-4579-85b6-33a1b0b555f0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:13.149809 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1065/2140] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:01:12 2022] POST /identity/v3/projects => generated 403 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:13.164069 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b1282-3040-473d-bf90-971879d6d343 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:13.260979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1a40a6ce-8356-4579-85b6-33a1b0b555f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ea242e7b6e3431ca5fe7ccff1108ffe', 'request_id': 'req-1a40a6ce-8356-4579-85b6-33a1b0b555f0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:13.283856 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1a40a6ce-8356-4579-85b6-33a1b0b555f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ea242e7b6e3431ca5fe7ccff1108ffe', 'request_id': 'req-1a40a6ce-8356-4579-85b6-33a1b0b555f0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:13.301276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1a40a6ce-8356-4579-85b6-33a1b0b555f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ea242e7b6e3431ca5fe7ccff1108ffe', 'request_id': 'req-1a40a6ce-8356-4579-85b6-33a1b0b555f0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:13.338265 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:13.382114 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1a40a6ce-8356-4579-85b6-33a1b0b555f0 admin admin] The token cache is being invalidate because project 6ea242e7b6e3431ca5fe7ccff1108ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:01:13.382439 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1a40a6ce-8356-4579-85b6-33a1b0b555f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:13.387001 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1a40a6ce-8356-4579-85b6-33a1b0b555f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:13.389218 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1076/2141] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:01:13 2022] DELETE /identity/v3/projects/6ea242e7b6e3431ca5fe7ccff1108ffe => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:13.394538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391b277-c4f9-4cf2-9494-8370c280db83 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:13.394849 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391b277-c4f9-4cf2-9494-8370c280db83 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:13.395135 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391b277-c4f9-4cf2-9494-8370c280db83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:13.467403 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b1282-3040-473d-bf90-971879d6d343 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:13.493728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0391b277-c4f9-4cf2-9494-8370c280db83 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:13.499443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0391b277-c4f9-4cf2-9494-8370c280db83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:13.711602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9cb556-7f3a-4ec9-99c0-584eade5d118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:13.712396 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cb556-7f3a-4ec9-99c0-584eade5d118 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:13.712578 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cb556-7f3a-4ec9-99c0-584eade5d118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:13.712858 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cb556-7f3a-4ec9-99c0-584eade5d118 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:13.742105 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9cb556-7f3a-4ec9-99c0-584eade5d118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:13.745449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1077/2142] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:13.751881 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c94530-beb6-400f-9a92-395da92aeba3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:13.752563 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c94530-beb6-400f-9a92-395da92aeba3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:13.752902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c94530-beb6-400f-9a92-395da92aeba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:13.764034 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9cb556-7f3a-4ec9-99c0-584eade5d118 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:13.802433 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-64c94530-beb6-400f-9a92-395da92aeba3 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:13.812625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-64c94530-beb6-400f-9a92-395da92aeba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:13.853840 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1078/2143] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:13.859792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1066/2144] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:01:13 2022] POST /identity/v3/users => generated 302 bytes in 696 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:13.861263 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c94530-beb6-400f-9a92-395da92aeba3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:13.864537 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9cb556-7f3a-4ec9-99c0-584eade5d118 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:13.866586 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9e9cb556-7f3a-4ec9-99c0-584eade5d118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:13.903540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c94530-beb6-400f-9a92-395da92aeba3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:13.907156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9cb556-7f3a-4ec9-99c0-584eade5d118 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:13.932817 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2424c-2dc5-4bea-998c-d8117b7a33d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:13.933577 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2424c-2dc5-4bea-998c-d8117b7a33d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:13.933934 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2424c-2dc5-4bea-998c-d8117b7a33d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:13.934379 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2424c-2dc5-4bea-998c-d8117b7a33d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:13.940081 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2424c-2dc5-4bea-998c-d8117b7a33d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:13.940081 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2424c-2dc5-4bea-998c-d8117b7a33d4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:13.952211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840f12a-fd09-498e-b8b4-1dcc3747a442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:13.953076 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840f12a-fd09-498e-b8b4-1dcc3747a442 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:13.953076 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840f12a-fd09-498e-b8b4-1dcc3747a442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:13.953702 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840f12a-fd09-498e-b8b4-1dcc3747a442 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:13.954497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840f12a-fd09-498e-b8b4-1dcc3747a442 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:01:13.954883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840f12a-fd09-498e-b8b4-1dcc3747a442 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:13.958928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1067/2145] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:13.963983 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2424c-2dc5-4bea-998c-d8117b7a33d4 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:13.967057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840f12a-fd09-498e-b8b4-1dcc3747a442 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:14.005406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1079/2146] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:01:14.009739 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840f12a-fd09-498e-b8b4-1dcc3747a442 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:14.010433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2424c-2dc5-4bea-998c-d8117b7a33d4 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:14.030033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5789d48-00c8-4e30-ad79-41d074da0a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:14.031058 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5789d48-00c8-4e30-ad79-41d074da0a09 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:14.031360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5789d48-00c8-4e30-ad79-41d074da0a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:14.032707 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5789d48-00c8-4e30-ad79-41d074da0a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:14.037231 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5789d48-00c8-4e30-ad79-41d074da0a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:14.037792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5789d48-00c8-4e30-ad79-41d074da0a09 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:14.063278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840f12a-fd09-498e-b8b4-1dcc3747a442 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:14.080141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1068/2147] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:14.083533 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8624c-c0a7-484b-97c3-fbbf637e93ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:14.083835 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5789d48-00c8-4e30-ad79-41d074da0a09 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:14.084270 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8624c-c0a7-484b-97c3-fbbf637e93ac admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:14.084599 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8624c-c0a7-484b-97c3-fbbf637e93ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:14.084882 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8624c-c0a7-484b-97c3-fbbf637e93ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:14.085667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8624c-c0a7-484b-97c3-fbbf637e93ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:14.096575 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8624c-c0a7-484b-97c3-fbbf637e93ac admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:14.108300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5789d48-00c8-4e30-ad79-41d074da0a09 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:14.120949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2e547-b27c-4ed0-925d-5641a1fe8593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:14.122469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2e547-b27c-4ed0-925d-5641a1fe8593 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:14.122870 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2e547-b27c-4ed0-925d-5641a1fe8593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:14.123398 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2e547-b27c-4ed0-925d-5641a1fe8593 admin admin] PATH_INFO: `/v3/projects/95360c0d4f114a0792d10011a38f0701/users/2e8e82c4ae9d4f72bd19ff12c12aa9df/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:14.165729 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2e547-b27c-4ed0-925d-5641a1fe8593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95360c0d4f114a0792d10011a38f0701, user_id=2e8e82c4ae9d4f72bd19ff12c12aa9df, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:14.169771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2e547-b27c-4ed0-925d-5641a1fe8593 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:14.204983 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1069/2148] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:14 2022] PUT /identity/v3/projects/95360c0d4f114a0792d10011a38f0701/users/2e8e82c4ae9d4f72bd19ff12c12aa9df/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:14.206884 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb8a3c-5d83-4097-b5e6-26abf02ba8f8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:14.207398 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb8a3c-5d83-4097-b5e6-26abf02ba8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:14.207815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb8a3c-5d83-4097-b5e6-26abf02ba8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:14.214014 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1080/2149] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:01:14 2022] POST /identity/v3/projects => generated 379 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:01:14.218317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8624c-c0a7-484b-97c3-fbbf637e93ac admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:14.252212 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-8afb8a3c-5d83-4097-b5e6-26abf02ba8f8 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:14.255636 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8afb8a3c-5d83-4097-b5e6-26abf02ba8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:14.302251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:14.426108 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1070/2150] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:01:14.449246 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb8a3c-5d83-4097-b5e6-26abf02ba8f8 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:14.455836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8afb8a3c-5d83-4097-b5e6-26abf02ba8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:14.498385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8624c-c0a7-484b-97c3-fbbf637e93ac admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:14.503047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:14.518600 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee04aec-ba63-4b6e-b59f-3706f764e34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:14.519462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee04aec-ba63-4b6e-b59f-3706f764e34b admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:14.519833 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee04aec-ba63-4b6e-b59f-3706f764e34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:14.520208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee04aec-ba63-4b6e-b59f-3706f764e34b admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:14.527650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee04aec-ba63-4b6e-b59f-3706f764e34b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:14.529792 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee04aec-ba63-4b6e-b59f-3706f764e34b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:14.598200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1081/2151] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:01:14 2022] POST /identity/v3/users => generated 290 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:14.602204 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee04aec-ba63-4b6e-b59f-3706f764e34b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:14.633368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee04aec-ba63-4b6e-b59f-3706f764e34b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:14.646141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189389c4-a067-45c6-81d4-afe2046c4745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:14.646938 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189389c4-a067-45c6-81d4-afe2046c4745 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:14.647323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189389c4-a067-45c6-81d4-afe2046c4745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:14.647654 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189389c4-a067-45c6-81d4-afe2046c4745 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:14.648756 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189389c4-a067-45c6-81d4-afe2046c4745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:14.649712 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb8a3c-5d83-4097-b5e6-26abf02ba8f8 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:14.651765 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189389c4-a067-45c6-81d4-afe2046c4745 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:14.656648 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782a024-0e9c-4890-a46c-5e7e35933d63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:14.657936 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782a024-0e9c-4890-a46c-5e7e35933d63 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:14.658599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782a024-0e9c-4890-a46c-5e7e35933d63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:14.659260 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782a024-0e9c-4890-a46c-5e7e35933d63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:14.676528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782a024-0e9c-4890-a46c-5e7e35933d63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:14.683006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1082/2152] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:14.683970 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782a024-0e9c-4890-a46c-5e7e35933d63 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:14.685579 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189389c4-a067-45c6-81d4-afe2046c4745 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:14.705850 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189389c4-a067-45c6-81d4-afe2046c4745 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:14.713978 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7effba-5205-479c-a052-0a0481833168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:14.714818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7effba-5205-479c-a052-0a0481833168 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:14.715215 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7effba-5205-479c-a052-0a0481833168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:14.716842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7effba-5205-479c-a052-0a0481833168 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:14.717738 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7effba-5205-479c-a052-0a0481833168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:14.719864 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1071/2153] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:14.720415 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7effba-5205-479c-a052-0a0481833168 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:14.742583 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1083/2154] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:01:14.744187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782a024-0e9c-4890-a46c-5e7e35933d63 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:14.760877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782a024-0e9c-4890-a46c-5e7e35933d63 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:14.791197 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a4515-73d6-4342-b2a2-fdb31519ebf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:14.792330 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a4515-73d6-4342-b2a2-fdb31519ebf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:14.792705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a4515-73d6-4342-b2a2-fdb31519ebf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:14.793099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a4515-73d6-4342-b2a2-fdb31519ebf5 admin admin] PATH_INFO: `/v3/projects/0e52189be8d24c9894ef05cc7acf367c/users/922a970a740444a2959c6aa1d0921ee7/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:14.833458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a4515-73d6-4342-b2a2-fdb31519ebf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e52189be8d24c9894ef05cc7acf367c, user_id=922a970a740444a2959c6aa1d0921ee7, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:14.844248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a4515-73d6-4342-b2a2-fdb31519ebf5 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:14.922447 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1072/2155] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:14 2022] PUT /identity/v3/projects/0e52189be8d24c9894ef05cc7acf367c/users/922a970a740444a2959c6aa1d0921ee7/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:01:14.927794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611709cc-ebce-4268-8018-bfc8627018d7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:14.928343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611709cc-ebce-4268-8018-bfc8627018d7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:14.928343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611709cc-ebce-4268-8018-bfc8627018d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:14.973499 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-611709cc-ebce-4268-8018-bfc8627018d7 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:14.985417 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-611709cc-ebce-4268-8018-bfc8627018d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:15.239243 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1084/2156] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:15.275670 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a4515-73d6-4342-b2a2-fdb31519ebf5 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:15.278715 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-df3a4515-73d6-4342-b2a2-fdb31519ebf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:15.413471 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a4515-73d6-4342-b2a2-fdb31519ebf5 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:15.424754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98964c6-8a6d-489b-8f10-b99205c5e74a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:15.425518 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98964c6-8a6d-489b-8f10-b99205c5e74a service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:15.425988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98964c6-8a6d-489b-8f10-b99205c5e74a service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:15.427680 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98964c6-8a6d-489b-8f10-b99205c5e74a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:15.456436 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98964c6-8a6d-489b-8f10-b99205c5e74a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:15.462016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98964c6-8a6d-489b-8f10-b99205c5e74a service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:15.509477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1073/2157] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:16.255958 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1a038-60be-43d1-a30e-bdfb265ad9a5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:16.257029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1a038-60be-43d1-a30e-bdfb265ad9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:16.257490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1a038-60be-43d1-a30e-bdfb265ad9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:16.296542 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1fb1a038-60be-43d1-a30e-bdfb265ad9a5 None None] MFA Rules not processed for user `2e8e82c4ae9d4f72bd19ff12c12aa9df`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:16.298295 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1fb1a038-60be-43d1-a30e-bdfb265ad9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:16.384173 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1085/2158] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:16.390304 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98964c6-8a6d-489b-8f10-b99205c5e74a service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:16.409314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98964c6-8a6d-489b-8f10-b99205c5e74a service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:16.456649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b6e96-a970-43d1-ad22-23991b02d36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:16.461181 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b6e96-a970-43d1-ad22-23991b02d36a admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:16.462294 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b6e96-a970-43d1-ad22-23991b02d36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:16.473665 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b6e96-a970-43d1-ad22-23991b02d36a admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:16.484796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b6e96-a970-43d1-ad22-23991b02d36a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:16.498913 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b6e96-a970-43d1-ad22-23991b02d36a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:16.581226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1074/2159] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:01:16 2022] POST /identity/v3/projects => generated 403 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:16.583326 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1a038-60be-43d1-a30e-bdfb265ad9a5 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:16.599692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1a038-60be-43d1-a30e-bdfb265ad9a5 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:16.622924 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:16.774120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88546a0c-55a1-451e-a863-c6c6ddbde1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:16.776821 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88546a0c-55a1-451e-a863-c6c6ddbde1c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:16.777396 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88546a0c-55a1-451e-a863-c6c6ddbde1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:16.778689 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88546a0c-55a1-451e-a863-c6c6ddbde1c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:16.781764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88546a0c-55a1-451e-a863-c6c6ddbde1c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:16.786436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88546a0c-55a1-451e-a863-c6c6ddbde1c0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:16.843423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1086/2160] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:01:16 2022] POST /identity/v3/users => generated 308 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:16.849967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b6e96-a970-43d1-ad22-23991b02d36a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:16.869500 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b6e96-a970-43d1-ad22-23991b02d36a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:16.886479 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae7246-c878-4ac7-b03c-4732c6330ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:16.888492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae7246-c878-4ac7-b03c-4732c6330ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:16.888798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae7246-c878-4ac7-b03c-4732c6330ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:16.889126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae7246-c878-4ac7-b03c-4732c6330ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:16.890074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ae7246-c878-4ac7-b03c-4732c6330ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:16.901874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ae7246-c878-4ac7-b03c-4732c6330ee6 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:16.942657 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1075/2161] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:16.951742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88546a0c-55a1-451e-a863-c6c6ddbde1c0 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:16.976946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88546a0c-55a1-451e-a863-c6c6ddbde1c0 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:16.989696 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce1fd2-69cb-462d-93e7-92223a403641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:16.991199 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce1fd2-69cb-462d-93e7-92223a403641 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:16.991866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce1fd2-69cb-462d-93e7-92223a403641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:16.992283 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce1fd2-69cb-462d-93e7-92223a403641 admin admin] PATH_INFO: `/v3/projects/97f52837adf1469c96714d1a37eeec75/users/b386583f55d141db807bd44e57c71314/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:17.058029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ce1fd2-69cb-462d-93e7-92223a403641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97f52837adf1469c96714d1a37eeec75, user_id=b386583f55d141db807bd44e57c71314, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:17.064879 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ce1fd2-69cb-462d-93e7-92223a403641 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:17.097968 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1087/2162] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:16 2022] PUT /identity/v3/projects/97f52837adf1469c96714d1a37eeec75/users/b386583f55d141db807bd44e57c71314/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:17.101596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae7246-c878-4ac7-b03c-4732c6330ee6 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:17.120202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae7246-c878-4ac7-b03c-4732c6330ee6 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:17.211900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:17.283445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbaf403-0424-445e-ba44-d63a13775f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:17.284621 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaf403-0424-445e-ba44-d63a13775f67 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:17.284911 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaf403-0424-445e-ba44-d63a13775f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:17.285193 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaf403-0424-445e-ba44-d63a13775f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:17.286093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbaf403-0424-445e-ba44-d63a13775f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:17.287681 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbaf403-0424-445e-ba44-d63a13775f67 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:17.314544 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1076/2163] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:17.315706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce1fd2-69cb-462d-93e7-92223a403641 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:17.334281 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce1fd2-69cb-462d-93e7-92223a403641 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:17.341755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92333cb9-b1aa-4a4a-944b-a3b9f0784785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:17.343133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92333cb9-b1aa-4a4a-944b-a3b9f0784785 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:17.343658 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92333cb9-b1aa-4a4a-944b-a3b9f0784785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:17.344184 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92333cb9-b1aa-4a4a-944b-a3b9f0784785 admin admin] PATH_INFO: `/v3/projects/97f52837adf1469c96714d1a37eeec75/users/b386583f55d141db807bd44e57c71314/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:17.348196 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92333cb9-b1aa-4a4a-944b-a3b9f0784785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97f52837adf1469c96714d1a37eeec75, user_id=b386583f55d141db807bd44e57c71314, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:17.351406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92333cb9-b1aa-4a4a-944b-a3b9f0784785 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:17.373754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1088/2164] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:17 2022] PUT /identity/v3/projects/97f52837adf1469c96714d1a37eeec75/users/b386583f55d141db807bd44e57c71314/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:01:19.930167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa3162-8e07-4461-a1e9-16186f59e111 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:19.931255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa3162-8e07-4461-a1e9-16186f59e111 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:19.931802 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa3162-8e07-4461-a1e9-16186f59e111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:20.049573 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-58aa3162-8e07-4461-a1e9-16186f59e111 None None] MFA Rules not processed for user `b386583f55d141db807bd44e57c71314`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:20.055319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-58aa3162-8e07-4461-a1e9-16186f59e111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:20.228918 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1077/2165] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:19 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:21.183313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a61ad2-e4ed-4ed8-9d01-43541df5d5a3 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:21.184702 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a61ad2-e4ed-4ed8-9d01-43541df5d5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:21.185859 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a61ad2-e4ed-4ed8-9d01-43541df5d5a3 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:21.187507 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1089/2166] 10.222.0.70 () {54 vars in 921 bytes} [Thu Jun 16 19:01:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:01:21.201621 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b277bf-861a-4868-8286-38a497e848b7 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:21.202126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b277bf-861a-4868-8286-38a497e848b7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:21.202397 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b277bf-861a-4868-8286-38a497e848b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:21.247327 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-11b277bf-861a-4868-8286-38a497e848b7 None None] MFA Rules not processed for user `880d0299c38c4eb897c669a512e599be`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:21.249377 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-11b277bf-861a-4868-8286-38a497e848b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:21.343983 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1078/2167] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Jun 16 19:01:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:01:21.499343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a61ad2-e4ed-4ed8-9d01-43541df5d5a3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:21.501159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a6a61ad2-e4ed-4ed8-9d01-43541df5d5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:21.580347 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a61ad2-e4ed-4ed8-9d01-43541df5d5a3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:21.591543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6b1c3-5018-4a95-a20d-fce64abb582b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '880d0299c38c4eb897c669a512e599be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:21.593313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6b1c3-5018-4a95-a20d-fce64abb582b service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:21.593621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6b1c3-5018-4a95-a20d-fce64abb582b service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:21.598076 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6b1c3-5018-4a95-a20d-fce64abb582b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:21.611365 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6b1c3-5018-4a95-a20d-fce64abb582b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:21.613862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6b1c3-5018-4a95-a20d-fce64abb582b service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:21.649388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1090/2168] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:22.945627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b277bf-861a-4868-8286-38a497e848b7 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:23.132405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:23.254277 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b277bf-861a-4868-8286-38a497e848b7 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:23.274945 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4a4f2-8bce-40d4-a66d-7659162a7b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:23.280053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4a4f2-8bce-40d4-a66d-7659162a7b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:23.281266 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4a4f2-8bce-40d4-a66d-7659162a7b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:23.281813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4a4f2-8bce-40d4-a66d-7659162a7b85 admin admin] PATH_INFO: `/v3/users/2e8e82c4ae9d4f72bd19ff12c12aa9df` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:23.290724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4a4f2-8bce-40d4-a66d-7659162a7b85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e8e82c4ae9d4f72bd19ff12c12aa9df)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:23.301351 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4a4f2-8bce-40d4-a66d-7659162a7b85 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:23.381906 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2fe4a4f2-8bce-40d4-a66d-7659162a7b85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e8e82c4ae9d4f72bd19ff12c12aa9df'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:23.383771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2fe4a4f2-8bce-40d4-a66d-7659162a7b85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e8e82c4ae9d4f72bd19ff12c12aa9df'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:23.395945 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2fe4a4f2-8bce-40d4-a66d-7659162a7b85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e8e82c4ae9d4f72bd19ff12c12aa9df'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:23.417348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2fe4a4f2-8bce-40d4-a66d-7659162a7b85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e8e82c4ae9d4f72bd19ff12c12aa9df'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:23.420001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2fe4a4f2-8bce-40d4-a66d-7659162a7b85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e8e82c4ae9d4f72bd19ff12c12aa9df'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:23.454746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1079/2169] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:01:22 2022] DELETE /identity/v3/users/2e8e82c4ae9d4f72bd19ff12c12aa9df => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:23.831174 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6b1c3-5018-4a95-a20d-fce64abb582b service placement] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:23.854514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6b1c3-5018-4a95-a20d-fce64abb582b service placement] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:24.001357 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a1feb-c9fb-42d6-883d-be382d65a214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:24.003558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a1feb-c9fb-42d6-883d-be382d65a214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:24.003851 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a1feb-c9fb-42d6-883d-be382d65a214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:24.004134 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a1feb-c9fb-42d6-883d-be382d65a214 admin admin] PATH_INFO: `/v3/projects/95360c0d4f114a0792d10011a38f0701` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:24.007053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a1feb-c9fb-42d6-883d-be382d65a214 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95360c0d4f114a0792d10011a38f0701)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:24.009364 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a1feb-c9fb-42d6-883d-be382d65a214 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:24.093525 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-429a1feb-c9fb-42d6-883d-be382d65a214 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95360c0d4f114a0792d10011a38f0701', 'request_id': 'req-429a1feb-c9fb-42d6-883d-be382d65a214'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:24.104437 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-429a1feb-c9fb-42d6-883d-be382d65a214 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95360c0d4f114a0792d10011a38f0701', 'request_id': 'req-429a1feb-c9fb-42d6-883d-be382d65a214'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:24.107403 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:24.128941 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-429a1feb-c9fb-42d6-883d-be382d65a214 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95360c0d4f114a0792d10011a38f0701', 'request_id': 'req-429a1feb-c9fb-42d6-883d-be382d65a214'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:24.155675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-429a1feb-c9fb-42d6-883d-be382d65a214 admin admin] The token cache is being invalidate because project 95360c0d4f114a0792d10011a38f0701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:01:24.157120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-429a1feb-c9fb-42d6-883d-be382d65a214 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:24.159898 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-429a1feb-c9fb-42d6-883d-be382d65a214 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:24.161603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1091/2170] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:01:23 2022] DELETE /identity/v3/projects/95360c0d4f114a0792d10011a38f0701 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:01:24.165621 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4a4f2-8bce-40d4-a66d-7659162a7b85 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:24.167418 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2fe4a4f2-8bce-40d4-a66d-7659162a7b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:24.187449 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4a4f2-8bce-40d4-a66d-7659162a7b85 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:24.316036 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab629046-c88a-42de-b1c1-aa1159226388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:24.317487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab629046-c88a-42de-b1c1-aa1159226388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:24.318081 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab629046-c88a-42de-b1c1-aa1159226388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:24.318540 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab629046-c88a-42de-b1c1-aa1159226388 admin admin] PATH_INFO: `/v3/users/b386583f55d141db807bd44e57c71314` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:24.320326 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab629046-c88a-42de-b1c1-aa1159226388 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b386583f55d141db807bd44e57c71314)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:24.326415 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab629046-c88a-42de-b1c1-aa1159226388 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:24.469513 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ab629046-c88a-42de-b1c1-aa1159226388 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b386583f55d141db807bd44e57c71314'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:24.470542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ab629046-c88a-42de-b1c1-aa1159226388 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b386583f55d141db807bd44e57c71314'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:24.488810 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ab629046-c88a-42de-b1c1-aa1159226388 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b386583f55d141db807bd44e57c71314'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:24.495706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:24.547759 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ab629046-c88a-42de-b1c1-aa1159226388 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b386583f55d141db807bd44e57c71314'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:24.549637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ab629046-c88a-42de-b1c1-aa1159226388 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b386583f55d141db807bd44e57c71314'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:24.582037 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1080/2171] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:01:24 2022] DELETE /identity/v3/users/b386583f55d141db807bd44e57c71314 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:24.978045 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a1feb-c9fb-42d6-883d-be382d65a214 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:24.980306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-429a1feb-c9fb-42d6-883d-be382d65a214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:25.001782 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a1feb-c9fb-42d6-883d-be382d65a214 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:25.114836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec23a-19fd-43fd-bd45-01398692f856 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:25.115432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec23a-19fd-43fd-bd45-01398692f856 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:25.115964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec23a-19fd-43fd-bd45-01398692f856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:25.142992 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a4d32-3b63-48ae-8520-9d1991bae700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:25.144107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a4d32-3b63-48ae-8520-9d1991bae700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:25.144512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a4d32-3b63-48ae-8520-9d1991bae700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:25.144797 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a4d32-3b63-48ae-8520-9d1991bae700 admin admin] PATH_INFO: `/v3/projects/97f52837adf1469c96714d1a37eeec75` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:25.147194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a4d32-3b63-48ae-8520-9d1991bae700 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97f52837adf1469c96714d1a37eeec75)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:25.151801 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a4d32-3b63-48ae-8520-9d1991bae700 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:25.183955 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f17ec23a-19fd-43fd-bd45-01398692f856 None None] MFA Rules not processed for user `922a970a740444a2959c6aa1d0921ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:25.188742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f17ec23a-19fd-43fd-bd45-01398692f856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:25.289890 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-922a4d32-3b63-48ae-8520-9d1991bae700 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97f52837adf1469c96714d1a37eeec75', 'request_id': 'req-922a4d32-3b63-48ae-8520-9d1991bae700'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:25.311276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-922a4d32-3b63-48ae-8520-9d1991bae700 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97f52837adf1469c96714d1a37eeec75', 'request_id': 'req-922a4d32-3b63-48ae-8520-9d1991bae700'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:25.319992 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-922a4d32-3b63-48ae-8520-9d1991bae700 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97f52837adf1469c96714d1a37eeec75', 'request_id': 'req-922a4d32-3b63-48ae-8520-9d1991bae700'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:25.348011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-922a4d32-3b63-48ae-8520-9d1991bae700 admin admin] The token cache is being invalidate because project 97f52837adf1469c96714d1a37eeec75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:01:25.348692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-922a4d32-3b63-48ae-8520-9d1991bae700 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:25.349477 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-922a4d32-3b63-48ae-8520-9d1991bae700 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:25.359111 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1092/2172] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:01:24 2022] DELETE /identity/v3/projects/97f52837adf1469c96714d1a37eeec75 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:25.361551 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc0e3c-3cb9-4829-ad27-3f87cdb78e08 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:25.364095 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc0e3c-3cb9-4829-ad27-3f87cdb78e08 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:25.364095 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc0e3c-3cb9-4829-ad27-3f87cdb78e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:25.409257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1081/2173] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:25.410518 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ec23a-19fd-43fd-bd45-01398692f856 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:25.414459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f17ec23a-19fd-43fd-bd45-01398692f856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:25.443279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ec23a-19fd-43fd-bd45-01398692f856 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:25.450458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-09cc0e3c-3cb9-4829-ad27-3f87cdb78e08 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:25.452799 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-09cc0e3c-3cb9-4829-ad27-3f87cdb78e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:25.542864 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:25.561988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80c08f-bc3e-48d0-b427-36767b895ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:25.563172 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80c08f-bc3e-48d0-b427-36767b895ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:25.564784 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80c08f-bc3e-48d0-b427-36767b895ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:25.566070 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80c08f-bc3e-48d0-b427-36767b895ed3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:25.567570 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80c08f-bc3e-48d0-b427-36767b895ed3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:25.571417 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80c08f-bc3e-48d0-b427-36767b895ed3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:25.582847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1093/2174] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:25.584647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093187d-f0d6-43d2-ade5-e0f287045b40 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:25.584928 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093187d-f0d6-43d2-ade5-e0f287045b40 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:25.585232 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093187d-f0d6-43d2-ade5-e0f287045b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:25.619074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1082/2175] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:01:25 2022] POST /identity/v3/projects => generated 379 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:25.623907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4093187d-f0d6-43d2-ade5-e0f287045b40 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:25.625025 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80c08f-bc3e-48d0-b427-36767b895ed3 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:25.626380 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4093187d-f0d6-43d2-ade5-e0f287045b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:25.760727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:25.786385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1094/2176] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:25.792110 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093187d-f0d6-43d2-ade5-e0f287045b40 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:25.805851 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80c08f-bc3e-48d0-b427-36767b895ed3 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:25.851120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093187d-f0d6-43d2-ade5-e0f287045b40 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:25.873536 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7aada-adeb-48a2-b732-f1b05677f5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:25.925515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7aada-adeb-48a2-b732-f1b05677f5a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:25.925883 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7aada-adeb-48a2-b732-f1b05677f5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:25.926082 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7aada-adeb-48a2-b732-f1b05677f5a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:25.933319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7aada-adeb-48a2-b732-f1b05677f5a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:25.979308 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb23d2b-56ad-4969-829c-814b0f4d77ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:25.986351 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb23d2b-56ad-4969-829c-814b0f4d77ef admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:25.992138 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb23d2b-56ad-4969-829c-814b0f4d77ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:25.992138 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb23d2b-56ad-4969-829c-814b0f4d77ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:25.992426 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7aada-adeb-48a2-b732-f1b05677f5a5 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:26.026920 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb23d2b-56ad-4969-829c-814b0f4d77ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:01:26.071436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb23d2b-56ad-4969-829c-814b0f4d77ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:26.083828 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb23d2b-56ad-4969-829c-814b0f4d77ef admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:26.139197 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1095/2177] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:01:26.148427 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb23d2b-56ad-4969-829c-814b0f4d77ef admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:26.154287 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1083/2178] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:01:25 2022] POST /identity/v3/users => generated 296 bytes in 526 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:26.156764 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7aada-adeb-48a2-b732-f1b05677f5a5 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:26.207963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb23d2b-56ad-4969-829c-814b0f4d77ef admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:26.211377 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7aada-adeb-48a2-b732-f1b05677f5a5 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:26.233865 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907e217-29da-4fe6-9f29-9609e5c239a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:26.234475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907e217-29da-4fe6-9f29-9609e5c239a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:26.234790 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907e217-29da-4fe6-9f29-9609e5c239a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:26.241392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907e217-29da-4fe6-9f29-9609e5c239a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:26.242305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907e217-29da-4fe6-9f29-9609e5c239a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:26.244527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907e217-29da-4fe6-9f29-9609e5c239a8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:26.246598 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c58fca-7462-4c23-8033-4ada0b9b5e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:26.247400 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c58fca-7462-4c23-8033-4ada0b9b5e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:26.247668 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c58fca-7462-4c23-8033-4ada0b9b5e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:26.248067 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c58fca-7462-4c23-8033-4ada0b9b5e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:26.250113 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c58fca-7462-4c23-8033-4ada0b9b5e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:26.251624 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c58fca-7462-4c23-8033-4ada0b9b5e8b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:26.296551 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1084/2179] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:26.297508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c58fca-7462-4c23-8033-4ada0b9b5e8b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:26.321114 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1096/2180] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:01:26 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:26.322827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907e217-29da-4fe6-9f29-9609e5c239a8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:26.385049 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c58fca-7462-4c23-8033-4ada0b9b5e8b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:26.485310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907e217-29da-4fe6-9f29-9609e5c239a8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:26.498904 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a6283-afc0-4163-8fe6-22543723024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:26.499983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a6283-afc0-4163-8fe6-22543723024d admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:26.501152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a6283-afc0-4163-8fe6-22543723024d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:26.501445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a6283-afc0-4163-8fe6-22543723024d admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:26.502265 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a6283-afc0-4163-8fe6-22543723024d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:26.506438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a6283-afc0-4163-8fe6-22543723024d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:26.549234 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1097/2181] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:01:26 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:26.550588 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a6283-afc0-4163-8fe6-22543723024d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:26.566859 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a6283-afc0-4163-8fe6-22543723024d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:26.571995 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d1821-e9f9-4561-a85d-cd714b9cf112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:26.573049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47767f-918c-456d-8d8d-b81527d51740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:26.574028 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d1821-e9f9-4561-a85d-cd714b9cf112 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:26.574154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47767f-918c-456d-8d8d-b81527d51740 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:26.575211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47767f-918c-456d-8d8d-b81527d51740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:26.575860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47767f-918c-456d-8d8d-b81527d51740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:26.576062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d1821-e9f9-4561-a85d-cd714b9cf112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:26.576421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d1821-e9f9-4561-a85d-cd714b9cf112 admin admin] PATH_INFO: `/v3/projects/210593d00452434e9eb7568a4084fc0b/users/9af6d57157e74a83a9776604dfd86f50/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:26.577389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47767f-918c-456d-8d8d-b81527d51740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:26.579661 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47767f-918c-456d-8d8d-b81527d51740 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:26.597759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1098/2182] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:26.602007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47767f-918c-456d-8d8d-b81527d51740 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:26.624609 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47767f-918c-456d-8d8d-b81527d51740 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:26.629935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d1821-e9f9-4561-a85d-cd714b9cf112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=210593d00452434e9eb7568a4084fc0b, user_id=9af6d57157e74a83a9776604dfd86f50, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:26.634692 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d1821-e9f9-4561-a85d-cd714b9cf112 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:26.639476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313fa6a-539e-403e-ac6a-c5b6f92fa19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:26.640691 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313fa6a-539e-403e-ac6a-c5b6f92fa19b admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:26.640979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313fa6a-539e-403e-ac6a-c5b6f92fa19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:26.641268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313fa6a-539e-403e-ac6a-c5b6f92fa19b admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:26.645385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313fa6a-539e-403e-ac6a-c5b6f92fa19b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:26.654556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313fa6a-539e-403e-ac6a-c5b6f92fa19b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:26.659458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1085/2183] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:26 2022] PUT /identity/v3/projects/210593d00452434e9eb7568a4084fc0b/users/9af6d57157e74a83a9776604dfd86f50/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:26.664744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d1821-e9f9-4561-a85d-cd714b9cf112 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:26.687644 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1099/2184] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:01:26.692367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313fa6a-539e-403e-ac6a-c5b6f92fa19b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:26.715222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313fa6a-539e-403e-ac6a-c5b6f92fa19b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:26.761004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:26.842294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc59712-a876-4503-b6ea-fb67c41d318c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:26.843382 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc59712-a876-4503-b6ea-fb67c41d318c admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:26.843755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc59712-a876-4503-b6ea-fb67c41d318c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:26.844038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc59712-a876-4503-b6ea-fb67c41d318c admin admin] PATH_INFO: `/v3/projects/ab99299c9cb54b398e5b9dd1757a6274/users/f03ef1eadc60471e992895dbfd1d328c/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:26.864895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d1821-e9f9-4561-a85d-cd714b9cf112 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:26.866182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc59712-a876-4503-b6ea-fb67c41d318c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab99299c9cb54b398e5b9dd1757a6274, user_id=f03ef1eadc60471e992895dbfd1d328c, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:26.869095 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc59712-a876-4503-b6ea-fb67c41d318c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:26.874268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702bf9d-bf19-4f40-a31d-b41472339f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:26.875848 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702bf9d-bf19-4f40-a31d-b41472339f0e admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:26.875848 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702bf9d-bf19-4f40-a31d-b41472339f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:26.875848 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702bf9d-bf19-4f40-a31d-b41472339f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:26.876845 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702bf9d-bf19-4f40-a31d-b41472339f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:26.882739 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702bf9d-bf19-4f40-a31d-b41472339f0e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:26.908209 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1100/2185] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:26 2022] PUT /identity/v3/projects/ab99299c9cb54b398e5b9dd1757a6274/users/f03ef1eadc60471e992895dbfd1d328c/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:26.921118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b63caf-2c87-4b58-bbf8-1cae3bf779f3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:26.921437 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b63caf-2c87-4b58-bbf8-1cae3bf779f3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:26.921737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b63caf-2c87-4b58-bbf8-1cae3bf779f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:26.922004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1086/2186] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:26.923052 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702bf9d-bf19-4f40-a31d-b41472339f0e admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:26.939301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702bf9d-bf19-4f40-a31d-b41472339f0e admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:27.036165 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-73b63caf-2c87-4b58-bbf8-1cae3bf779f3 None None] MFA Rules not processed for user `f03ef1eadc60471e992895dbfd1d328c`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:27.037546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-73b63caf-2c87-4b58-bbf8-1cae3bf779f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:27.101184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:27.133019 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0cc1a-6b40-4f66-89db-be2f38906c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:27.133440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0cc1a-6b40-4f66-89db-be2f38906c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:27.133802 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0cc1a-6b40-4f66-89db-be2f38906c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:27.133978 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0cc1a-6b40-4f66-89db-be2f38906c53 admin admin] PATH_INFO: `/v3/projects/210593d00452434e9eb7568a4084fc0b/users/9af6d57157e74a83a9776604dfd86f50/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:27.149714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0cc1a-6b40-4f66-89db-be2f38906c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=210593d00452434e9eb7568a4084fc0b, user_id=9af6d57157e74a83a9776604dfd86f50, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:27.154872 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0cc1a-6b40-4f66-89db-be2f38906c53 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:27.184132 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1101/2187] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:27.186291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b63caf-2c87-4b58-bbf8-1cae3bf779f3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:27.197016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1087/2188] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:26 2022] PUT /identity/v3/projects/210593d00452434e9eb7568a4084fc0b/users/9af6d57157e74a83a9776604dfd86f50/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:27.211577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b63caf-2c87-4b58-bbf8-1cae3bf779f3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:27.395240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa07418-9c0d-48cd-b064-231e6c866f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:27.397731 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa07418-9c0d-48cd-b064-231e6c866f72 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:27.399034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa07418-9c0d-48cd-b064-231e6c866f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:27.400000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa07418-9c0d-48cd-b064-231e6c866f72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:27.403460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa07418-9c0d-48cd-b064-231e6c866f72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:27.406129 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa07418-9c0d-48cd-b064-231e6c866f72 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:27.460888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1102/2189] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:01:27 2022] POST /identity/v3/projects => generated 371 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:01:27.464184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0cc1a-6b40-4f66-89db-be2f38906c53 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:27.481494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0cc1a-6b40-4f66-89db-be2f38906c53 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:27.749413 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4ddfd-c494-4472-a465-e431ea4bd0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:27.752765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4ddfd-c494-4472-a465-e431ea4bd0ea admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:27.755471 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4ddfd-c494-4472-a465-e431ea4bd0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:27.755778 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4ddfd-c494-4472-a465-e431ea4bd0ea admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:27.756604 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d4ddfd-c494-4472-a465-e431ea4bd0ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:27.758932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d4ddfd-c494-4472-a465-e431ea4bd0ea admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:27.824920 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1088/2190] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:01:27 2022] POST /identity/v3/users => generated 292 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:27.834621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa07418-9c0d-48cd-b064-231e6c866f72 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:27.855451 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa07418-9c0d-48cd-b064-231e6c866f72 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:27.865851 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229fb01-8844-467e-a588-380981c1bd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:27.866538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229fb01-8844-467e-a588-380981c1bd54 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:27.866847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229fb01-8844-467e-a588-380981c1bd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:27.867120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229fb01-8844-467e-a588-380981c1bd54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:27.867849 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229fb01-8844-467e-a588-380981c1bd54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:27.870875 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229fb01-8844-467e-a588-380981c1bd54 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:27.907944 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1103/2191] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:27.911993 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4ddfd-c494-4472-a465-e431ea4bd0ea admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:27.925553 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4ddfd-c494-4472-a465-e431ea4bd0ea admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:27.938465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414388b-6fa8-4800-936b-5b4d78b35675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:27.939404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414388b-6fa8-4800-936b-5b4d78b35675 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:27.939689 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414388b-6fa8-4800-936b-5b4d78b35675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:27.939969 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414388b-6fa8-4800-936b-5b4d78b35675 admin admin] PATH_INFO: `/v3/projects/4c68d186f9114234a400a2d361f53ced/users/43ac584a6596473e93d15cc6b6484751/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:27.988506 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414388b-6fa8-4800-936b-5b4d78b35675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c68d186f9114234a400a2d361f53ced, user_id=43ac584a6596473e93d15cc6b6484751, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:27.991227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414388b-6fa8-4800-936b-5b4d78b35675 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:28.016545 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1089/2192] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:27 2022] PUT /identity/v3/projects/4c68d186f9114234a400a2d361f53ced/users/43ac584a6596473e93d15cc6b6484751/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:28.022513 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229fb01-8844-467e-a588-380981c1bd54 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:28.043216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229fb01-8844-467e-a588-380981c1bd54 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:28.073081 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:28.223757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4727982-3f76-48bf-b85a-35cd9c8ac70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:28.224454 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4727982-3f76-48bf-b85a-35cd9c8ac70d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:28.224743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4727982-3f76-48bf-b85a-35cd9c8ac70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:28.225026 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4727982-3f76-48bf-b85a-35cd9c8ac70d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:28.225766 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4727982-3f76-48bf-b85a-35cd9c8ac70d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:28.236032 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4727982-3f76-48bf-b85a-35cd9c8ac70d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:28.252765 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1104/2193] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:28.258547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414388b-6fa8-4800-936b-5b4d78b35675 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:28.279220 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414388b-6fa8-4800-936b-5b4d78b35675 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:28.296630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4269ff0-ea67-4369-adc6-a08eeb63a27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:28.297788 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4269ff0-ea67-4369-adc6-a08eeb63a27f admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:28.298221 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4269ff0-ea67-4369-adc6-a08eeb63a27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:28.298827 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4269ff0-ea67-4369-adc6-a08eeb63a27f admin admin] PATH_INFO: `/v3/projects/4c68d186f9114234a400a2d361f53ced/users/43ac584a6596473e93d15cc6b6484751/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:28.315986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4269ff0-ea67-4369-adc6-a08eeb63a27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c68d186f9114234a400a2d361f53ced, user_id=43ac584a6596473e93d15cc6b6484751, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:28.328468 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4269ff0-ea67-4369-adc6-a08eeb63a27f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:28.371896 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1090/2194] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:28 2022] PUT /identity/v3/projects/4c68d186f9114234a400a2d361f53ced/users/43ac584a6596473e93d15cc6b6484751/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:28.382985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770c913-1465-4f46-b7f0-f8f217d75896 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:28.383311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770c913-1465-4f46-b7f0-f8f217d75896 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:28.383586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770c913-1465-4f46-b7f0-f8f217d75896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:28.509721 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b770c913-1465-4f46-b7f0-f8f217d75896 None None] MFA Rules not processed for user `43ac584a6596473e93d15cc6b6484751`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:28.511021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b770c913-1465-4f46-b7f0-f8f217d75896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:28.792981 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1105/2195] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:28 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:28.803782 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4269ff0-ea67-4369-adc6-a08eeb63a27f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:28.805812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e4269ff0-ea67-4369-adc6-a08eeb63a27f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:29.054614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4269ff0-ea67-4369-adc6-a08eeb63a27f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:29.096071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8e8bf-7da9-4c5f-ac6a-e58cd8dfa26e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:29.096807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8e8bf-7da9-4c5f-ac6a-e58cd8dfa26e service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:29.097100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8e8bf-7da9-4c5f-ac6a-e58cd8dfa26e service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:29.097408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8e8bf-7da9-4c5f-ac6a-e58cd8dfa26e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:29.139715 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8e8bf-7da9-4c5f-ac6a-e58cd8dfa26e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:29.141341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8e8bf-7da9-4c5f-ac6a-e58cd8dfa26e service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:29.162084 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1091/2196] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:28 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:29.912815 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770c913-1465-4f46-b7f0-f8f217d75896 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:30.071705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770c913-1465-4f46-b7f0-f8f217d75896 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:30.093739 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c78fa4-e2a4-4547-8307-c23aefc77b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:30.094713 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c78fa4-e2a4-4547-8307-c23aefc77b65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:30.101027 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c78fa4-e2a4-4547-8307-c23aefc77b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:30.101718 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c78fa4-e2a4-4547-8307-c23aefc77b65 admin admin] PATH_INFO: `/v3/users/f03ef1eadc60471e992895dbfd1d328c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:30.104147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c78fa4-e2a4-4547-8307-c23aefc77b65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f03ef1eadc60471e992895dbfd1d328c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:30.108007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c78fa4-e2a4-4547-8307-c23aefc77b65 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:30.216581 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:30.238286 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-25c78fa4-e2a4-4547-8307-c23aefc77b65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f03ef1eadc60471e992895dbfd1d328c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:30.239458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-25c78fa4-e2a4-4547-8307-c23aefc77b65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f03ef1eadc60471e992895dbfd1d328c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:30.242186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-25c78fa4-e2a4-4547-8307-c23aefc77b65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f03ef1eadc60471e992895dbfd1d328c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:30.260144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-25c78fa4-e2a4-4547-8307-c23aefc77b65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f03ef1eadc60471e992895dbfd1d328c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:30.278031 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-25c78fa4-e2a4-4547-8307-c23aefc77b65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f03ef1eadc60471e992895dbfd1d328c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:30.338891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1106/2197] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:01:29 2022] DELETE /identity/v3/users/f03ef1eadc60471e992895dbfd1d328c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:30.344843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc810da6-1ac3-4f90-92ed-431458a94125 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:30.345762 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc810da6-1ac3-4f90-92ed-431458a94125 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:30.346233 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc810da6-1ac3-4f90-92ed-431458a94125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:30.384648 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-fc810da6-1ac3-4f90-92ed-431458a94125 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:30.386083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-fc810da6-1ac3-4f90-92ed-431458a94125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:30.487833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:30.570735 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1092/2198] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:30.603667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c78fa4-e2a4-4547-8307-c23aefc77b65 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:30.603667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-25c78fa4-e2a4-4547-8307-c23aefc77b65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:30.778836 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c78fa4-e2a4-4547-8307-c23aefc77b65 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:30.787468 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992dfe12-8c32-4e3a-b87d-1180a84e3f27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:30.788316 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dfe12-8c32-4e3a-b87d-1180a84e3f27 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:30.788711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dfe12-8c32-4e3a-b87d-1180a84e3f27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:30.789120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dfe12-8c32-4e3a-b87d-1180a84e3f27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:30.834606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992dfe12-8c32-4e3a-b87d-1180a84e3f27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:30.836675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992dfe12-8c32-4e3a-b87d-1180a84e3f27 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:30.881773 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1107/2199] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:32.154132 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228ab2b-4c7d-4361-9133-a656303dc40c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:32.155374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228ab2b-4c7d-4361-9133-a656303dc40c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:32.155881 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228ab2b-4c7d-4361-9133-a656303dc40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:32.163942 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992dfe12-8c32-4e3a-b87d-1180a84e3f27 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:32.165432 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-992dfe12-8c32-4e3a-b87d-1180a84e3f27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:32.195102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992dfe12-8c32-4e3a-b87d-1180a84e3f27 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:32.234460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:32.235569 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:32.235857 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:32.236136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec admin admin] PATH_INFO: `/v3/projects/ab99299c9cb54b398e5b9dd1757a6274` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:32.248819 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab99299c9cb54b398e5b9dd1757a6274)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:32.250871 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:32.270927 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4228ab2b-4c7d-4361-9133-a656303dc40c None None] MFA Rules not processed for user `7f703f74c7ff43a291f54103e164ccdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:32.272452 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4228ab2b-4c7d-4361-9133-a656303dc40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:32.603077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab99299c9cb54b398e5b9dd1757a6274', 'request_id': 'req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:32.642713 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab99299c9cb54b398e5b9dd1757a6274', 'request_id': 'req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:32.655015 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab99299c9cb54b398e5b9dd1757a6274', 'request_id': 'req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:32.682978 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1093/2200] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 533 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:32.694395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec admin admin] The token cache is being invalidate because project ab99299c9cb54b398e5b9dd1757a6274 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:01:32.695048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:32.703123 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:32.705167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1108/2201] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:01:32 2022] DELETE /identity/v3/projects/ab99299c9cb54b398e5b9dd1757a6274 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:32.709162 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228ab2b-4c7d-4361-9133-a656303dc40c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:32.729489 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228ab2b-4c7d-4361-9133-a656303dc40c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:32.873360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264cf3d-cc9e-4698-ab61-c5e592e0f18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:32.874542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264cf3d-cc9e-4698-ab61-c5e592e0f18c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:32.875052 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264cf3d-cc9e-4698-ab61-c5e592e0f18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:32.875508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264cf3d-cc9e-4698-ab61-c5e592e0f18c admin admin] PATH_INFO: `/v3/users/43ac584a6596473e93d15cc6b6484751` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:32.878510 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264cf3d-cc9e-4698-ab61-c5e592e0f18c admin admin] RBAC: Authorizing `identity:delete_user(user_id=43ac584a6596473e93d15cc6b6484751)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:32.880707 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264cf3d-cc9e-4698-ab61-c5e592e0f18c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:33.026498 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3264cf3d-cc9e-4698-ab61-c5e592e0f18c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43ac584a6596473e93d15cc6b6484751'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:33.028111 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3264cf3d-cc9e-4698-ab61-c5e592e0f18c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43ac584a6596473e93d15cc6b6484751'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:33.041612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3264cf3d-cc9e-4698-ab61-c5e592e0f18c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43ac584a6596473e93d15cc6b6484751'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:33.083461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3264cf3d-cc9e-4698-ab61-c5e592e0f18c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43ac584a6596473e93d15cc6b6484751'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:33.084214 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3264cf3d-cc9e-4698-ab61-c5e592e0f18c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43ac584a6596473e93d15cc6b6484751'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:33.101269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:33.110604 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1094/2202] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:01:32 2022] DELETE /identity/v3/users/43ac584a6596473e93d15cc6b6484751 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:33.768622 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:33.770131 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:33.808454 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1d6f6-9375-4cb7-88ca-5958764f82ec admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:33.873447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:33.969136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6119aa-8ad7-44ec-b0b8-257bdadf03c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:33.970027 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6119aa-8ad7-44ec-b0b8-257bdadf03c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:33.970338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6119aa-8ad7-44ec-b0b8-257bdadf03c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:33.970613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6119aa-8ad7-44ec-b0b8-257bdadf03c6 admin admin] PATH_INFO: `/v3/projects/4c68d186f9114234a400a2d361f53ced` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:33.974019 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6119aa-8ad7-44ec-b0b8-257bdadf03c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c68d186f9114234a400a2d361f53ced)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:33.982072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6119aa-8ad7-44ec-b0b8-257bdadf03c6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:34.082803 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264cf3d-cc9e-4698-ab61-c5e592e0f18c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:34.084913 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3264cf3d-cc9e-4698-ab61-c5e592e0f18c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:34.128262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bb6119aa-8ad7-44ec-b0b8-257bdadf03c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c68d186f9114234a400a2d361f53ced', 'request_id': 'req-bb6119aa-8ad7-44ec-b0b8-257bdadf03c6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:34.143823 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bb6119aa-8ad7-44ec-b0b8-257bdadf03c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c68d186f9114234a400a2d361f53ced', 'request_id': 'req-bb6119aa-8ad7-44ec-b0b8-257bdadf03c6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:34.160976 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bb6119aa-8ad7-44ec-b0b8-257bdadf03c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c68d186f9114234a400a2d361f53ced', 'request_id': 'req-bb6119aa-8ad7-44ec-b0b8-257bdadf03c6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:34.183248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264cf3d-cc9e-4698-ab61-c5e592e0f18c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:34.203791 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bb6119aa-8ad7-44ec-b0b8-257bdadf03c6 admin admin] The token cache is being invalidate because project 4c68d186f9114234a400a2d361f53ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:01:34.204085 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bb6119aa-8ad7-44ec-b0b8-257bdadf03c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:34.204846 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bb6119aa-8ad7-44ec-b0b8-257bdadf03c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:34.206743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1109/2203] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:01:33 2022] DELETE /identity/v3/projects/4c68d186f9114234a400a2d361f53ced => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:34.212035 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac700301-8370-43d1-9e42-5b9f38bcde2c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:34.212576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac700301-8370-43d1-9e42-5b9f38bcde2c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:34.212857 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac700301-8370-43d1-9e42-5b9f38bcde2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:34.274137 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ac700301-8370-43d1-9e42-5b9f38bcde2c None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:34.276831 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ac700301-8370-43d1-9e42-5b9f38bcde2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:34.320370 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10ba05-968c-4a1d-9e8d-83af18a0d845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:34.322467 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10ba05-968c-4a1d-9e8d-83af18a0d845 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:34.323096 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10ba05-968c-4a1d-9e8d-83af18a0d845 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:34.323537 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10ba05-968c-4a1d-9e8d-83af18a0d845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:34.326734 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-fa10ba05-968c-4a1d-9e8d-83af18a0d845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:34.421458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1110/2204] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:34.437768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df120f-8aeb-411f-8622-2f3626c7d8ad None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:34.437768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df120f-8aeb-411f-8622-2f3626c7d8ad None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:34.437768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df120f-8aeb-411f-8622-2f3626c7d8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:34.475406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-09df120f-8aeb-411f-8622-2f3626c7d8ad None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:34.479777 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-09df120f-8aeb-411f-8622-2f3626c7d8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:34.491969 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10ba05-968c-4a1d-9e8d-83af18a0d845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:34.495157 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10ba05-968c-4a1d-9e8d-83af18a0d845 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:34.506747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1111/2205] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:34.518471 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1095/2206] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:34 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:01:34.520453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df120f-8aeb-411f-8622-2f3626c7d8ad None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:34.534714 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df120f-8aeb-411f-8622-2f3626c7d8ad None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:34.540317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11562908-dbe8-4f41-8d4d-6e1f4189ee96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:34.541059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11562908-dbe8-4f41-8d4d-6e1f4189ee96 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:34.541358 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11562908-dbe8-4f41-8d4d-6e1f4189ee96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:34.541631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11562908-dbe8-4f41-8d4d-6e1f4189ee96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:34.542378 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11562908-dbe8-4f41-8d4d-6e1f4189ee96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:01:34.542761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11562908-dbe8-4f41-8d4d-6e1f4189ee96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:34.546212 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11562908-dbe8-4f41-8d4d-6e1f4189ee96 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:34.566877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1112/2207] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:01:34.568255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10ba05-968c-4a1d-9e8d-83af18a0d845 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:34.585826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10ba05-968c-4a1d-9e8d-83af18a0d845 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:34.593483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d00429-cc4b-45d0-8c56-da7e36c9973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:34.594742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d00429-cc4b-45d0-8c56-da7e36c9973e admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:34.595238 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d00429-cc4b-45d0-8c56-da7e36c9973e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:34.595827 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d00429-cc4b-45d0-8c56-da7e36c9973e admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:34.596753 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d00429-cc4b-45d0-8c56-da7e36c9973e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:34.600591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d00429-cc4b-45d0-8c56-da7e36c9973e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:34.756073 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11562908-dbe8-4f41-8d4d-6e1f4189ee96 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:34.756304 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1096/2208] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:01:34 2022] POST /identity/v3/projects => generated 377 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:34.774361 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11562908-dbe8-4f41-8d4d-6e1f4189ee96 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:34.842266 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:34.886528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8bfc52-1bdb-443a-9947-0dac86d0ca99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:34.889615 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bfc52-1bdb-443a-9947-0dac86d0ca99 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:34.889918 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bfc52-1bdb-443a-9947-0dac86d0ca99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:34.890205 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bfc52-1bdb-443a-9947-0dac86d0ca99 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:34.890985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8bfc52-1bdb-443a-9947-0dac86d0ca99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:34.906415 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8bfc52-1bdb-443a-9947-0dac86d0ca99 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:34.978053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d00429-cc4b-45d0-8c56-da7e36c9973e admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:34.980041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1113/2209] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:01:34 2022] POST /identity/v3/users => generated 289 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:34.994294 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d00429-cc4b-45d0-8c56-da7e36c9973e admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:35.006178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea3998-de77-45aa-abca-4124351a960c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:35.006985 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea3998-de77-45aa-abca-4124351a960c admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:35.007275 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea3998-de77-45aa-abca-4124351a960c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:35.007557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea3998-de77-45aa-abca-4124351a960c admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:35.008296 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea3998-de77-45aa-abca-4124351a960c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:35.016188 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea3998-de77-45aa-abca-4124351a960c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:35.055988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1097/2210] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:01:35.060790 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8bfc52-1bdb-443a-9947-0dac86d0ca99 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:35.080508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8bfc52-1bdb-443a-9947-0dac86d0ca99 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:35.120155 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aea8d9-16ea-4d51-b317-2285417b1106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:35.120854 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aea8d9-16ea-4d51-b317-2285417b1106 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:35.121717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aea8d9-16ea-4d51-b317-2285417b1106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:35.122506 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aea8d9-16ea-4d51-b317-2285417b1106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:35.122819 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aea8d9-16ea-4d51-b317-2285417b1106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:35.124522 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aea8d9-16ea-4d51-b317-2285417b1106 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:35.146373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1114/2211] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:35.150362 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea3998-de77-45aa-abca-4124351a960c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:35.165257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea3998-de77-45aa-abca-4124351a960c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:35.174817 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740040e-6168-4408-8b76-c58dee526349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:35.177179 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740040e-6168-4408-8b76-c58dee526349 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:35.177470 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740040e-6168-4408-8b76-c58dee526349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:35.177815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740040e-6168-4408-8b76-c58dee526349 admin admin] PATH_INFO: `/v3/projects/d515754b455f4dd49d39a7120f33502b/users/2d74724c08ba422d94788fffc08fa1f2/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:35.197589 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740040e-6168-4408-8b76-c58dee526349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d515754b455f4dd49d39a7120f33502b, user_id=2d74724c08ba422d94788fffc08fa1f2, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:35.201001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740040e-6168-4408-8b76-c58dee526349 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:35.228828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1098/2212] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:35 2022] PUT /identity/v3/projects/d515754b455f4dd49d39a7120f33502b/users/2d74724c08ba422d94788fffc08fa1f2/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:35.242703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea3ded-665d-47fd-a8d9-4e7af7f2d426 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:35.243281 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea3ded-665d-47fd-a8d9-4e7af7f2d426 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:35.243903 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea3ded-665d-47fd-a8d9-4e7af7f2d426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:35.292633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c7ea3ded-665d-47fd-a8d9-4e7af7f2d426 None None] MFA Rules not processed for user `2d74724c08ba422d94788fffc08fa1f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:35.294425 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c7ea3ded-665d-47fd-a8d9-4e7af7f2d426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:35.435119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cbb8b-6427-49d2-835c-65df99d49dfb None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:35.435427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cbb8b-6427-49d2-835c-65df99d49dfb None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:35.435609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cbb8b-6427-49d2-835c-65df99d49dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:35.451431 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1115/2213] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:35.456042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea3ded-665d-47fd-a8d9-4e7af7f2d426 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:35.476366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-133cbb8b-6427-49d2-835c-65df99d49dfb None None] MFA Rules not processed for user `047e841cefdd4975947b2c154412953b`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:35.477616 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-133cbb8b-6427-49d2-835c-65df99d49dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:35.486645 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:35.564055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea3ded-665d-47fd-a8d9-4e7af7f2d426 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:35.574820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af55101-4440-4d76-a4e0-f949ff92dbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:35.575660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af55101-4440-4d76-a4e0-f949ff92dbde admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:35.575876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af55101-4440-4d76-a4e0-f949ff92dbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:35.576183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af55101-4440-4d76-a4e0-f949ff92dbde admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:35.576974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af55101-4440-4d76-a4e0-f949ff92dbde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:35.584972 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af55101-4440-4d76-a4e0-f949ff92dbde admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:35.588093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1099/2214] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:35.640811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1116/2215] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:01:35 2022] POST /identity/v3/projects => generated 379 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:35.645377 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133cbb8b-6427-49d2-835c-65df99d49dfb None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:35.669092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133cbb8b-6427-49d2-835c-65df99d49dfb None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:35.786565 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e3485-e49c-4fd9-a5bb-a5d32cc9b39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:35.787034 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e3485-e49c-4fd9-a5bb-a5d32cc9b39b admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:35.787375 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e3485-e49c-4fd9-a5bb-a5d32cc9b39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:35.787654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e3485-e49c-4fd9-a5bb-a5d32cc9b39b admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:35.794614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e3485-e49c-4fd9-a5bb-a5d32cc9b39b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:35.796852 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e3485-e49c-4fd9-a5bb-a5d32cc9b39b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:35.830685 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af55101-4440-4d76-a4e0-f949ff92dbde admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:35.843916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1100/2216] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:01:35 2022] POST /identity/v3/users => generated 290 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:35.847642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af55101-4440-4d76-a4e0-f949ff92dbde admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:35.873692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa166dc7-1172-4c54-8e4d-d99dd403bb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:35.873692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa166dc7-1172-4c54-8e4d-d99dd403bb14 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:35.873692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa166dc7-1172-4c54-8e4d-d99dd403bb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:35.873692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa166dc7-1172-4c54-8e4d-d99dd403bb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:35.874217 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa166dc7-1172-4c54-8e4d-d99dd403bb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:35.874217 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa166dc7-1172-4c54-8e4d-d99dd403bb14 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:35.886362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1117/2217] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:01:35.890696 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e3485-e49c-4fd9-a5bb-a5d32cc9b39b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:35.907093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e3485-e49c-4fd9-a5bb-a5d32cc9b39b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:35.913097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14313e47-fbda-4de9-a2f4-010e9bf60c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:35.914890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14313e47-fbda-4de9-a2f4-010e9bf60c83 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:35.915802 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14313e47-fbda-4de9-a2f4-010e9bf60c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:35.916246 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14313e47-fbda-4de9-a2f4-010e9bf60c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:35.917149 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14313e47-fbda-4de9-a2f4-010e9bf60c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:35.921444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14313e47-fbda-4de9-a2f4-010e9bf60c83 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:35.939958 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1101/2218] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:01:35.943473 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa166dc7-1172-4c54-8e4d-d99dd403bb14 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:35.969203 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa166dc7-1172-4c54-8e4d-d99dd403bb14 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:35.976198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cce48-4d53-4f00-ad43-20091e21c55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:35.981416 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cce48-4d53-4f00-ad43-20091e21c55c admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:35.981726 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cce48-4d53-4f00-ad43-20091e21c55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:35.982555 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cce48-4d53-4f00-ad43-20091e21c55c admin admin] PATH_INFO: `/v3/projects/a93f7bae5f3c414a985fd4d1480f9be4/users/88ebcc5cb6f44fa1b46fb792050c69c5/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:36.030173 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:36.044285 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8cce48-4d53-4f00-ad43-20091e21c55c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a93f7bae5f3c414a985fd4d1480f9be4, user_id=88ebcc5cb6f44fa1b46fb792050c69c5, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:36.050292 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8cce48-4d53-4f00-ad43-20091e21c55c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:36.081266 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1118/2219] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:35 2022] PUT /identity/v3/projects/a93f7bae5f3c414a985fd4d1480f9be4/users/88ebcc5cb6f44fa1b46fb792050c69c5/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:01:36.092437 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb30e3b-8a9c-431f-9b00-b31aad1f9bc7 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:36.092750 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb30e3b-8a9c-431f-9b00-b31aad1f9bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:36.093104 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb30e3b-8a9c-431f-9b00-b31aad1f9bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:36.131060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-dcb30e3b-8a9c-431f-9b00-b31aad1f9bc7 None None] MFA Rules not processed for user `88ebcc5cb6f44fa1b46fb792050c69c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:36.132378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-dcb30e3b-8a9c-431f-9b00-b31aad1f9bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:36.282705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1102/2220] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:36.287467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cce48-4d53-4f00-ad43-20091e21c55c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:36.494000 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cce48-4d53-4f00-ad43-20091e21c55c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:36.509051 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e7d13-776a-49eb-90b8-41490f0c915a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:36.510110 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e7d13-776a-49eb-90b8-41490f0c915a admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:36.510411 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e7d13-776a-49eb-90b8-41490f0c915a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:36.510792 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e7d13-776a-49eb-90b8-41490f0c915a admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:36.511642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e7d13-776a-49eb-90b8-41490f0c915a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:36.517660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e7d13-776a-49eb-90b8-41490f0c915a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:36.590924 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1119/2221] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:01:36 2022] POST /identity/v3/projects => generated 379 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:36.595189 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb30e3b-8a9c-431f-9b00-b31aad1f9bc7 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:36.613289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb30e3b-8a9c-431f-9b00-b31aad1f9bc7 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:36.673946 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:36.752322 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2c8b8-7afd-407c-980b-9c73a0dbf1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:36.754096 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2c8b8-7afd-407c-980b-9c73a0dbf1ac admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:36.754664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2c8b8-7afd-407c-980b-9c73a0dbf1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:36.755157 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2c8b8-7afd-407c-980b-9c73a0dbf1ac admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:36.756388 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2c8b8-7afd-407c-980b-9c73a0dbf1ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:36.759329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2c8b8-7afd-407c-980b-9c73a0dbf1ac admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:36.805354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1103/2222] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:01:36 2022] POST /identity/v3/users => generated 296 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:36.805760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e7d13-776a-49eb-90b8-41490f0c915a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:36.831001 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e7d13-776a-49eb-90b8-41490f0c915a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:36.849291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d335bb-3ded-4a07-a0fe-7d984d3339f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:36.853080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d335bb-3ded-4a07-a0fe-7d984d3339f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:36.853810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d335bb-3ded-4a07-a0fe-7d984d3339f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:36.854114 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d335bb-3ded-4a07-a0fe-7d984d3339f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:36.854902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d335bb-3ded-4a07-a0fe-7d984d3339f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:36.861789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d335bb-3ded-4a07-a0fe-7d984d3339f0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:36.895892 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1120/2223] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:01:36.899988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2c8b8-7afd-407c-980b-9c73a0dbf1ac admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:36.918849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2c8b8-7afd-407c-980b-9c73a0dbf1ac admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:36.934936 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155598f-8c57-4195-a16d-5bf2824f4d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:36.940253 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155598f-8c57-4195-a16d-5bf2824f4d78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:36.941062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155598f-8c57-4195-a16d-5bf2824f4d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:36.952949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155598f-8c57-4195-a16d-5bf2824f4d78 admin admin] PATH_INFO: `/v3/projects/8bddaedbaef44a8aa5b881c4bea1ca3a/users/0708286e05dc48d5a814c02e3966a0c3/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:37.000618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155598f-8c57-4195-a16d-5bf2824f4d78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bddaedbaef44a8aa5b881c4bea1ca3a, user_id=0708286e05dc48d5a814c02e3966a0c3, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:37.016182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155598f-8c57-4195-a16d-5bf2824f4d78 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:37.082505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1104/2224] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:36 2022] PUT /identity/v3/projects/8bddaedbaef44a8aa5b881c4bea1ca3a/users/0708286e05dc48d5a814c02e3966a0c3/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:37.086723 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d335bb-3ded-4a07-a0fe-7d984d3339f0 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:37.109033 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d335bb-3ded-4a07-a0fe-7d984d3339f0 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:37.417672 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f44fc-163e-4484-8218-71faedc722f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:37.420843 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f44fc-163e-4484-8218-71faedc722f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:37.422742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f44fc-163e-4484-8218-71faedc722f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:37.423888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f44fc-163e-4484-8218-71faedc722f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:37.424862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f44fc-163e-4484-8218-71faedc722f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:37.439040 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f44fc-163e-4484-8218-71faedc722f8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:37.488901 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155598f-8c57-4195-a16d-5bf2824f4d78 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:37.489744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1121/2225] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:37.509052 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155598f-8c57-4195-a16d-5bf2824f4d78 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:37.531792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce3af9-be0c-461b-abb8-05924fa6b7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:37.532762 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce3af9-be0c-461b-abb8-05924fa6b7e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:37.533018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce3af9-be0c-461b-abb8-05924fa6b7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:37.535627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce3af9-be0c-461b-abb8-05924fa6b7e8 admin admin] PATH_INFO: `/v3/projects/8bddaedbaef44a8aa5b881c4bea1ca3a/users/0708286e05dc48d5a814c02e3966a0c3/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:37.543895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce3af9-be0c-461b-abb8-05924fa6b7e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bddaedbaef44a8aa5b881c4bea1ca3a, user_id=0708286e05dc48d5a814c02e3966a0c3, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:37.555386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce3af9-be0c-461b-abb8-05924fa6b7e8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:37.609179 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1105/2226] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:37 2022] PUT /identity/v3/projects/8bddaedbaef44a8aa5b881c4bea1ca3a/users/0708286e05dc48d5a814c02e3966a0c3/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:01:37.623709 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb5ef7-2dfa-4a14-ad53-85fff43fb411 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:37.623883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb5ef7-2dfa-4a14-ad53-85fff43fb411 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:37.624698 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb5ef7-2dfa-4a14-ad53-85fff43fb411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:37.716660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f5eb5ef7-2dfa-4a14-ad53-85fff43fb411 None None] MFA Rules not processed for user `0708286e05dc48d5a814c02e3966a0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:37.719885 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f5eb5ef7-2dfa-4a14-ad53-85fff43fb411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:37.892022 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:37.989081 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1122/2227] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:37 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:38.001798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce3af9-be0c-461b-abb8-05924fa6b7e8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:38.004153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f7ce3af9-be0c-461b-abb8-05924fa6b7e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:38.150494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce3af9-be0c-461b-abb8-05924fa6b7e8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:38.162314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02d440-fd65-4f09-aedb-9d545267afa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:38.163364 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d440-fd65-4f09-aedb-9d545267afa9 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:38.166093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d440-fd65-4f09-aedb-9d545267afa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:38.166754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d440-fd65-4f09-aedb-9d545267afa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:38.201595 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02d440-fd65-4f09-aedb-9d545267afa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:38.204866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02d440-fd65-4f09-aedb-9d545267afa9 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:38.245396 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1106/2228] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:38.328599 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb5ef7-2dfa-4a14-ad53-85fff43fb411 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:38.335459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f5eb5ef7-2dfa-4a14-ad53-85fff43fb411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:38.388694 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb5ef7-2dfa-4a14-ad53-85fff43fb411 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:38.411687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c042f-72dd-45e6-ab39-c06dc52b37f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:38.415486 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c042f-72dd-45e6-ab39-c06dc52b37f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:38.416241 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c042f-72dd-45e6-ab39-c06dc52b37f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:38.416856 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c042f-72dd-45e6-ab39-c06dc52b37f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:38.608830 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c042f-72dd-45e6-ab39-c06dc52b37f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:38.623216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c042f-72dd-45e6-ab39-c06dc52b37f1 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:38.662344 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1123/2229] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:01:39.449787 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6f575-bb9b-41ac-bdc1-93ba40980cc8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:39.451194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6f575-bb9b-41ac-bdc1-93ba40980cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:39.451845 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6f575-bb9b-41ac-bdc1-93ba40980cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:39.493379 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f0b6f575-bb9b-41ac-bdc1-93ba40980cc8 None None] MFA Rules not processed for user `9af6d57157e74a83a9776604dfd86f50`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:39.495126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f0b6f575-bb9b-41ac-bdc1-93ba40980cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:39.692822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c042f-72dd-45e6-ab39-c06dc52b37f1 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:39.695615 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-032c042f-72dd-45e6-ab39-c06dc52b37f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:39.697187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1107/2230] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:39.884080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c042f-72dd-45e6-ab39-c06dc52b37f1 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:39.897366 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91ea78-0a5e-42ae-aaa1-1b47bbf97266 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:39.899651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91ea78-0a5e-42ae-aaa1-1b47bbf97266 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:39.903102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91ea78-0a5e-42ae-aaa1-1b47bbf97266 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:39.903501 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91ea78-0a5e-42ae-aaa1-1b47bbf97266 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:39.907873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1b91ea78-0a5e-42ae-aaa1-1b47bbf97266 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:40.094049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91ea78-0a5e-42ae-aaa1-1b47bbf97266 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:40.095937 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91ea78-0a5e-42ae-aaa1-1b47bbf97266 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:40.163355 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1124/2231] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:01:40.408513 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4fe90-1338-49be-8a9f-887b99cf31e6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:40.409334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4fe90-1338-49be-8a9f-887b99cf31e6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:40.409768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4fe90-1338-49be-8a9f-887b99cf31e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:40.416311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:40.484251 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f8d4fe90-1338-49be-8a9f-887b99cf31e6 None None] MFA Rules not processed for user `02c826fc0baa480393df4c884c9e35dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:40.485663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f8d4fe90-1338-49be-8a9f-887b99cf31e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:40.620234 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1108/2232] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Jun 16 19:01:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:40.641802 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91ea78-0a5e-42ae-aaa1-1b47bbf97266 service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:40.643570 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1b91ea78-0a5e-42ae-aaa1-1b47bbf97266 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:40.696977 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:40.785056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91ea78-0a5e-42ae-aaa1-1b47bbf97266 service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:40.820293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d10750-2acc-44f3-9ec9-443124d2e505 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b22fe0ab0354385924923d0d443acae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:40.821064 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d10750-2acc-44f3-9ec9-443124d2e505 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:40.821468 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d10750-2acc-44f3-9ec9-443124d2e505 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:40.821687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d10750-2acc-44f3-9ec9-443124d2e505 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:40.840341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d10750-2acc-44f3-9ec9-443124d2e505 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:40.842736 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d10750-2acc-44f3-9ec9-443124d2e505 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:40.874747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1125/2233] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Jun 16 19:01:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:01:43.079972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4fe90-1338-49be-8a9f-887b99cf31e6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:43.317935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4fe90-1338-49be-8a9f-887b99cf31e6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:43.334940 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659a1c0-be30-43d2-ad8d-caa1b5b2a0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:43.338525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659a1c0-be30-43d2-ad8d-caa1b5b2a0f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:43.338889 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659a1c0-be30-43d2-ad8d-caa1b5b2a0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:43.339192 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659a1c0-be30-43d2-ad8d-caa1b5b2a0f2 admin admin] PATH_INFO: `/v3/users/2d74724c08ba422d94788fffc08fa1f2` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:43.341841 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659a1c0-be30-43d2-ad8d-caa1b5b2a0f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d74724c08ba422d94788fffc08fa1f2)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:43.353379 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659a1c0-be30-43d2-ad8d-caa1b5b2a0f2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:43.528837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d659a1c0-be30-43d2-ad8d-caa1b5b2a0f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d74724c08ba422d94788fffc08fa1f2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:43.529771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d659a1c0-be30-43d2-ad8d-caa1b5b2a0f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d74724c08ba422d94788fffc08fa1f2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:43.561703 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d659a1c0-be30-43d2-ad8d-caa1b5b2a0f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d74724c08ba422d94788fffc08fa1f2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:43.592912 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d659a1c0-be30-43d2-ad8d-caa1b5b2a0f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d74724c08ba422d94788fffc08fa1f2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:43.593919 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d659a1c0-be30-43d2-ad8d-caa1b5b2a0f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d74724c08ba422d94788fffc08fa1f2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:43.625550 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1109/2234] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:01:43 2022] DELETE /identity/v3/users/2d74724c08ba422d94788fffc08fa1f2 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:43.627440 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1385af-2946-46c8-b541-9449a460b73e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:43.627942 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1385af-2946-46c8-b541-9449a460b73e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:43.628026 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1385af-2946-46c8-b541-9449a460b73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:43.662780 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-eb1385af-2946-46c8-b541-9449a460b73e None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:43.673769 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-eb1385af-2946-46c8-b541-9449a460b73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:43.788232 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1126/2235] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:43.804361 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659a1c0-be30-43d2-ad8d-caa1b5b2a0f2 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:43.806129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d659a1c0-be30-43d2-ad8d-caa1b5b2a0f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:43.877513 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:43.953141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659a1c0-be30-43d2-ad8d-caa1b5b2a0f2 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:43.966161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad60e9-2403-46ba-90b7-88e5d06109bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:43.970968 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad60e9-2403-46ba-90b7-88e5d06109bc service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:43.971275 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad60e9-2403-46ba-90b7-88e5d06109bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:43.971646 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad60e9-2403-46ba-90b7-88e5d06109bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:43.990242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad60e9-2403-46ba-90b7-88e5d06109bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:43.991931 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad60e9-2403-46ba-90b7-88e5d06109bc service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:44.026197 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1110/2236] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:44.875735 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1385af-2946-46c8-b541-9449a460b73e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:44.878917 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-eb1385af-2946-46c8-b541-9449a460b73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:44.910082 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1385af-2946-46c8-b541-9449a460b73e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:44.937461 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7226be-b965-4324-92d5-b5bb484a49e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:44.938350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7226be-b965-4324-92d5-b5bb484a49e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:44.938898 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7226be-b965-4324-92d5-b5bb484a49e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:44.939184 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7226be-b965-4324-92d5-b5bb484a49e5 admin admin] PATH_INFO: `/v3/projects/d515754b455f4dd49d39a7120f33502b` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:44.948572 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7226be-b965-4324-92d5-b5bb484a49e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d515754b455f4dd49d39a7120f33502b)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:44.948572 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7226be-b965-4324-92d5-b5bb484a49e5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:45.191792 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ce7226be-b965-4324-92d5-b5bb484a49e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd515754b455f4dd49d39a7120f33502b', 'request_id': 'req-ce7226be-b965-4324-92d5-b5bb484a49e5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:45.225442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ce7226be-b965-4324-92d5-b5bb484a49e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd515754b455f4dd49d39a7120f33502b', 'request_id': 'req-ce7226be-b965-4324-92d5-b5bb484a49e5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:45.244930 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ce7226be-b965-4324-92d5-b5bb484a49e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd515754b455f4dd49d39a7120f33502b', 'request_id': 'req-ce7226be-b965-4324-92d5-b5bb484a49e5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:45.306120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ce7226be-b965-4324-92d5-b5bb484a49e5 admin admin] The token cache is being invalidate because project d515754b455f4dd49d39a7120f33502b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:01:45.306633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ce7226be-b965-4324-92d5-b5bb484a49e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:45.307580 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ce7226be-b965-4324-92d5-b5bb484a49e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:45.314760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1127/2237] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:01:44 2022] DELETE /identity/v3/projects/d515754b455f4dd49d39a7120f33502b => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:45.336060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad60e9-2403-46ba-90b7-88e5d06109bc service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:45.388814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad60e9-2403-46ba-90b7-88e5d06109bc service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:45.646593 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ad408-b934-486e-9496-8d27b540a473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:45.647844 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ad408-b934-486e-9496-8d27b540a473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:45.648361 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ad408-b934-486e-9496-8d27b540a473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:45.648860 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ad408-b934-486e-9496-8d27b540a473 admin admin] PATH_INFO: `/v3/users/88ebcc5cb6f44fa1b46fb792050c69c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:45.652943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ad408-b934-486e-9496-8d27b540a473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88ebcc5cb6f44fa1b46fb792050c69c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:45.674995 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ad408-b934-486e-9496-8d27b540a473 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:45.833161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6f6ad408-b934-486e-9496-8d27b540a473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88ebcc5cb6f44fa1b46fb792050c69c5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:45.834432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6f6ad408-b934-486e-9496-8d27b540a473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88ebcc5cb6f44fa1b46fb792050c69c5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:45.859149 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6f6ad408-b934-486e-9496-8d27b540a473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88ebcc5cb6f44fa1b46fb792050c69c5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:45.914864 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6f6ad408-b934-486e-9496-8d27b540a473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88ebcc5cb6f44fa1b46fb792050c69c5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:45.918681 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6f6ad408-b934-486e-9496-8d27b540a473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88ebcc5cb6f44fa1b46fb792050c69c5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:45.950906 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1111/2238] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:01:45 2022] DELETE /identity/v3/users/88ebcc5cb6f44fa1b46fb792050c69c5 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:46.719329 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7226be-b965-4324-92d5-b5bb484a49e5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:46.721289 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ce7226be-b965-4324-92d5-b5bb484a49e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:46.748883 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7226be-b965-4324-92d5-b5bb484a49e5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:46.814150 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:46.855234 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea267d-4389-40ba-b53e-d78851d82bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:46.856111 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea267d-4389-40ba-b53e-d78851d82bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:46.856699 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea267d-4389-40ba-b53e-d78851d82bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:46.857127 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea267d-4389-40ba-b53e-d78851d82bfe admin admin] PATH_INFO: `/v3/projects/a93f7bae5f3c414a985fd4d1480f9be4` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:46.859228 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea267d-4389-40ba-b53e-d78851d82bfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=a93f7bae5f3c414a985fd4d1480f9be4)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:46.863291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea267d-4389-40ba-b53e-d78851d82bfe admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:46.969753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8dea267d-4389-40ba-b53e-d78851d82bfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a93f7bae5f3c414a985fd4d1480f9be4', 'request_id': 'req-8dea267d-4389-40ba-b53e-d78851d82bfe'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:47.009245 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8dea267d-4389-40ba-b53e-d78851d82bfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a93f7bae5f3c414a985fd4d1480f9be4', 'request_id': 'req-8dea267d-4389-40ba-b53e-d78851d82bfe'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:47.021176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8dea267d-4389-40ba-b53e-d78851d82bfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a93f7bae5f3c414a985fd4d1480f9be4', 'request_id': 'req-8dea267d-4389-40ba-b53e-d78851d82bfe'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:47.057700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8dea267d-4389-40ba-b53e-d78851d82bfe admin admin] The token cache is being invalidate because project a93f7bae5f3c414a985fd4d1480f9be4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:01:47.060040 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8dea267d-4389-40ba-b53e-d78851d82bfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:47.062630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8dea267d-4389-40ba-b53e-d78851d82bfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:47.068943 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1128/2239] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:01:46 2022] DELETE /identity/v3/projects/a93f7bae5f3c414a985fd4d1480f9be4 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:47.073781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ad408-b934-486e-9496-8d27b540a473 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:47.078397 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6f6ad408-b934-486e-9496-8d27b540a473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:47.106020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ad408-b934-486e-9496-8d27b540a473 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:47.224724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:47.277015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4f357-45ba-4c03-8a5f-64e20bab4503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:47.278925 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4f357-45ba-4c03-8a5f-64e20bab4503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:47.279490 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4f357-45ba-4c03-8a5f-64e20bab4503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:47.280137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4f357-45ba-4c03-8a5f-64e20bab4503 admin admin] PATH_INFO: `/v3/users/0708286e05dc48d5a814c02e3966a0c3` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:47.286035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4f357-45ba-4c03-8a5f-64e20bab4503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0708286e05dc48d5a814c02e3966a0c3)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:47.291004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4f357-45ba-4c03-8a5f-64e20bab4503 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:47.400325 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-86f4f357-45ba-4c03-8a5f-64e20bab4503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0708286e05dc48d5a814c02e3966a0c3'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:47.403108 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-86f4f357-45ba-4c03-8a5f-64e20bab4503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0708286e05dc48d5a814c02e3966a0c3'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:47.430633 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-86f4f357-45ba-4c03-8a5f-64e20bab4503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0708286e05dc48d5a814c02e3966a0c3'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:47.459539 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-86f4f357-45ba-4c03-8a5f-64e20bab4503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0708286e05dc48d5a814c02e3966a0c3'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:47.460689 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-86f4f357-45ba-4c03-8a5f-64e20bab4503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0708286e05dc48d5a814c02e3966a0c3'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:47.484038 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1112/2240] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:01:47 2022] DELETE /identity/v3/users/0708286e05dc48d5a814c02e3966a0c3 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:01:48.062341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea267d-4389-40ba-b53e-d78851d82bfe admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:48.063818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8dea267d-4389-40ba-b53e-d78851d82bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:48.084674 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea267d-4389-40ba-b53e-d78851d82bfe admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:48.225418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc2bee-ab83-4096-a5dd-1d296c676775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:48.226296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc2bee-ab83-4096-a5dd-1d296c676775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:48.226615 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc2bee-ab83-4096-a5dd-1d296c676775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:48.226897 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc2bee-ab83-4096-a5dd-1d296c676775 admin admin] PATH_INFO: `/v3/projects/8bddaedbaef44a8aa5b881c4bea1ca3a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:48.228171 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc2bee-ab83-4096-a5dd-1d296c676775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bddaedbaef44a8aa5b881c4bea1ca3a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:48.231362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc2bee-ab83-4096-a5dd-1d296c676775 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:48.296582 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f3dc2bee-ab83-4096-a5dd-1d296c676775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bddaedbaef44a8aa5b881c4bea1ca3a', 'request_id': 'req-f3dc2bee-ab83-4096-a5dd-1d296c676775'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:48.304053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f3dc2bee-ab83-4096-a5dd-1d296c676775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bddaedbaef44a8aa5b881c4bea1ca3a', 'request_id': 'req-f3dc2bee-ab83-4096-a5dd-1d296c676775'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:48.311315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f3dc2bee-ab83-4096-a5dd-1d296c676775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bddaedbaef44a8aa5b881c4bea1ca3a', 'request_id': 'req-f3dc2bee-ab83-4096-a5dd-1d296c676775'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:48.316157 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:48.323521 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f3dc2bee-ab83-4096-a5dd-1d296c676775 admin admin] The token cache is being invalidate because project 8bddaedbaef44a8aa5b881c4bea1ca3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:01:48.324309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f3dc2bee-ab83-4096-a5dd-1d296c676775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:48.325776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f3dc2bee-ab83-4096-a5dd-1d296c676775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:48.327934 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1129/2241] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:01:48 2022] DELETE /identity/v3/projects/8bddaedbaef44a8aa5b881c4bea1ca3a => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:48.333739 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a6dd9-0fb2-41f4-bccf-5566408a55d5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:48.334305 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a6dd9-0fb2-41f4-bccf-5566408a55d5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:48.334406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a6dd9-0fb2-41f4-bccf-5566408a55d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:48.356119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-526a6dd9-0fb2-41f4-bccf-5566408a55d5 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:48.357222 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-526a6dd9-0fb2-41f4-bccf-5566408a55d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:48.425594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1113/2242] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:48.434153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee7310-8bf6-48bb-827e-a32b3dc74f0a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:48.434657 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee7310-8bf6-48bb-827e-a32b3dc74f0a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:48.434961 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee7310-8bf6-48bb-827e-a32b3dc74f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:48.453761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c8ee7310-8bf6-48bb-827e-a32b3dc74f0a None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:48.455004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c8ee7310-8bf6-48bb-827e-a32b3dc74f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:48.463367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1130/2243] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:48.468544 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a6dd9-0fb2-41f4-bccf-5566408a55d5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:48.478293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a6dd9-0fb2-41f4-bccf-5566408a55d5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:48.484067 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31911ec2-8321-4545-9233-7bef33b47ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:48.484900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31911ec2-8321-4545-9233-7bef33b47ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:48.485183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31911ec2-8321-4545-9233-7bef33b47ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:48.485460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31911ec2-8321-4545-9233-7bef33b47ee8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:48.486190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31911ec2-8321-4545-9233-7bef33b47ee8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:01:48.486629 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31911ec2-8321-4545-9233-7bef33b47ee8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:48.488432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31911ec2-8321-4545-9233-7bef33b47ee8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:48.501258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1114/2244] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:01:48.506095 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee7310-8bf6-48bb-827e-a32b3dc74f0a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:48.516496 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee7310-8bf6-48bb-827e-a32b3dc74f0a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:48.521579 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a947699-378d-4cb1-904c-60b988e96b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:48.522356 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a947699-378d-4cb1-904c-60b988e96b84 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:48.522909 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a947699-378d-4cb1-904c-60b988e96b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:48.523153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a947699-378d-4cb1-904c-60b988e96b84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:48.524588 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a947699-378d-4cb1-904c-60b988e96b84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:48.528970 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a947699-378d-4cb1-904c-60b988e96b84 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:48.587631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1131/2245] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:01:48 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:48.596486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31911ec2-8321-4545-9233-7bef33b47ee8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:48.608312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31911ec2-8321-4545-9233-7bef33b47ee8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:48.666779 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:48.682034 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac09efc-c30e-42f1-b395-9ee601f4cbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:48.682734 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac09efc-c30e-42f1-b395-9ee601f4cbf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:48.683101 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac09efc-c30e-42f1-b395-9ee601f4cbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:48.683382 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac09efc-c30e-42f1-b395-9ee601f4cbf9 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:48.684118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac09efc-c30e-42f1-b395-9ee601f4cbf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:48.686119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac09efc-c30e-42f1-b395-9ee601f4cbf9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:48.707387 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1115/2246] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:01:48 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:48.711272 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a947699-378d-4cb1-904c-60b988e96b84 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:48.724370 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a947699-378d-4cb1-904c-60b988e96b84 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:48.729565 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e913e-4043-4f67-99b3-06e197124eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:48.732475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e913e-4043-4f67-99b3-06e197124eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:48.732475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e913e-4043-4f67-99b3-06e197124eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:48.732475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e913e-4043-4f67-99b3-06e197124eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:48.732475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e913e-4043-4f67-99b3-06e197124eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:48.733843 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e913e-4043-4f67-99b3-06e197124eb8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:48.779206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1132/2247] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:48.783525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac09efc-c30e-42f1-b395-9ee601f4cbf9 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:48.799722 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac09efc-c30e-42f1-b395-9ee601f4cbf9 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:48.805177 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8a0ca-9ab1-4cbd-a63f-19fa1113af1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:48.806192 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8a0ca-9ab1-4cbd-a63f-19fa1113af1c admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:48.806485 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8a0ca-9ab1-4cbd-a63f-19fa1113af1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:48.806792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8a0ca-9ab1-4cbd-a63f-19fa1113af1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:48.807524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8a0ca-9ab1-4cbd-a63f-19fa1113af1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:48.810767 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8a0ca-9ab1-4cbd-a63f-19fa1113af1c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:48.841857 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1116/2248] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:48.846276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e913e-4043-4f67-99b3-06e197124eb8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:48.860774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e913e-4043-4f67-99b3-06e197124eb8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:48.865599 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8cb63-a265-4585-87a3-d6b043a24d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:48.866493 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8cb63-a265-4585-87a3-d6b043a24d57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:48.866809 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8cb63-a265-4585-87a3-d6b043a24d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:48.867088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8cb63-a265-4585-87a3-d6b043a24d57 admin admin] PATH_INFO: `/v3/projects/8e6a75170a0a4dea916f688ecc927b0a/users/d98f386fceaa44ea80c5e0b88928d4ea/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:48.887591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8cb63-a265-4585-87a3-d6b043a24d57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e6a75170a0a4dea916f688ecc927b0a, user_id=d98f386fceaa44ea80c5e0b88928d4ea, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:48.892560 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8cb63-a265-4585-87a3-d6b043a24d57 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:48.909215 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1133/2249] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:48 2022] PUT /identity/v3/projects/8e6a75170a0a4dea916f688ecc927b0a/users/d98f386fceaa44ea80c5e0b88928d4ea/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:48.924752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07d632-ee16-4511-9e4d-9b8313848732 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:48.925031 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07d632-ee16-4511-9e4d-9b8313848732 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:48.925315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07d632-ee16-4511-9e4d-9b8313848732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:48.962521 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-9c07d632-ee16-4511-9e4d-9b8313848732 None None] MFA Rules not processed for user `d98f386fceaa44ea80c5e0b88928d4ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:48.964055 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9c07d632-ee16-4511-9e4d-9b8313848732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:49.081696 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1117/2250] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:49.085445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8cb63-a265-4585-87a3-d6b043a24d57 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:49.189968 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8cb63-a265-4585-87a3-d6b043a24d57 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:49.197407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad0539-d614-45b7-bdc5-042a5cce0126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:49.198426 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad0539-d614-45b7-bdc5-042a5cce0126 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:49.199482 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad0539-d614-45b7-bdc5-042a5cce0126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:49.199847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad0539-d614-45b7-bdc5-042a5cce0126 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:49.201400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad0539-d614-45b7-bdc5-042a5cce0126 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:49.204278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad0539-d614-45b7-bdc5-042a5cce0126 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:49.253195 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1134/2251] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:01:49 2022] POST /identity/v3/projects => generated 383 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:01:49.254397 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07d632-ee16-4511-9e4d-9b8313848732 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:49.266530 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07d632-ee16-4511-9e4d-9b8313848732 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:49.344369 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d268b-cb81-465f-8aba-fb5e91c5a715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:49.345161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d268b-cb81-465f-8aba-fb5e91c5a715 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:49.345513 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d268b-cb81-465f-8aba-fb5e91c5a715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:49.345811 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d268b-cb81-465f-8aba-fb5e91c5a715 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:49.346471 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d268b-cb81-465f-8aba-fb5e91c5a715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:49.348651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d268b-cb81-465f-8aba-fb5e91c5a715 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:49.374393 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1118/2252] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:01:49 2022] POST /identity/v3/users => generated 298 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:49.378373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad0539-d614-45b7-bdc5-042a5cce0126 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:49.392990 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad0539-d614-45b7-bdc5-042a5cce0126 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:49.398113 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae956229-3780-48cf-9e1f-5398f15060bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:49.398794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae956229-3780-48cf-9e1f-5398f15060bc admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:49.399081 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae956229-3780-48cf-9e1f-5398f15060bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:49.399358 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae956229-3780-48cf-9e1f-5398f15060bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:49.400086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae956229-3780-48cf-9e1f-5398f15060bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:49.402681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae956229-3780-48cf-9e1f-5398f15060bc admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:49.415982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1135/2253] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:49.421012 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d268b-cb81-465f-8aba-fb5e91c5a715 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:49.437533 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d268b-cb81-465f-8aba-fb5e91c5a715 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:49.444716 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962558a-ff1d-4d0b-8bf6-465e5e8ddf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:49.445742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962558a-ff1d-4d0b-8bf6-465e5e8ddf56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:49.446100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962558a-ff1d-4d0b-8bf6-465e5e8ddf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:49.446451 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962558a-ff1d-4d0b-8bf6-465e5e8ddf56 admin admin] PATH_INFO: `/v3/projects/2689a520b57a432c893da8023605d4b0/users/dc50b516e27644a7910cd83a3632d657/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:49.464558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962558a-ff1d-4d0b-8bf6-465e5e8ddf56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2689a520b57a432c893da8023605d4b0, user_id=dc50b516e27644a7910cd83a3632d657, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:49.467860 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962558a-ff1d-4d0b-8bf6-465e5e8ddf56 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:49.488999 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1119/2254] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:49 2022] PUT /identity/v3/projects/2689a520b57a432c893da8023605d4b0/users/dc50b516e27644a7910cd83a3632d657/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:01:49.492781 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae956229-3780-48cf-9e1f-5398f15060bc admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:49.504082 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae956229-3780-48cf-9e1f-5398f15060bc admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:49.567271 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411cfd0-c820-42a2-8766-e01aa739cb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:49.568009 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411cfd0-c820-42a2-8766-e01aa739cb1d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:49.568459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411cfd0-c820-42a2-8766-e01aa739cb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:49.568742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411cfd0-c820-42a2-8766-e01aa739cb1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:49.569497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411cfd0-c820-42a2-8766-e01aa739cb1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:49.571364 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411cfd0-c820-42a2-8766-e01aa739cb1d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:49.586654 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1136/2255] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:49.594999 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962558a-ff1d-4d0b-8bf6-465e5e8ddf56 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:49.609547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962558a-ff1d-4d0b-8bf6-465e5e8ddf56 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:49.615253 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320a96e-10cb-42b9-b856-6067a0f96edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:49.616171 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320a96e-10cb-42b9-b856-6067a0f96edf admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:49.616449 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320a96e-10cb-42b9-b856-6067a0f96edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:49.616721 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320a96e-10cb-42b9-b856-6067a0f96edf admin admin] PATH_INFO: `/v3/projects/2689a520b57a432c893da8023605d4b0/users/dc50b516e27644a7910cd83a3632d657/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:49.618882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320a96e-10cb-42b9-b856-6067a0f96edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2689a520b57a432c893da8023605d4b0, user_id=dc50b516e27644a7910cd83a3632d657, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:49.623577 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320a96e-10cb-42b9-b856-6067a0f96edf admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:49.636016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1120/2256] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:49 2022] PUT /identity/v3/projects/2689a520b57a432c893da8023605d4b0/users/dc50b516e27644a7910cd83a3632d657/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:49.647368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd6a09-463e-4a34-a33b-e95216ee73f2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:49.647659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd6a09-463e-4a34-a33b-e95216ee73f2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:49.647946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd6a09-463e-4a34-a33b-e95216ee73f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:49.654565 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:49.691588 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e4dd6a09-463e-4a34-a33b-e95216ee73f2 None None] MFA Rules not processed for user `dc50b516e27644a7910cd83a3632d657`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:49.693078 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e4dd6a09-463e-4a34-a33b-e95216ee73f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:49.765287 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1137/2257] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:01:49.780594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320a96e-10cb-42b9-b856-6067a0f96edf admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:49.782607 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1320a96e-10cb-42b9-b856-6067a0f96edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:49.801758 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:49.922829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320a96e-10cb-42b9-b856-6067a0f96edf admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:49.926714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8731e6-13e6-42e6-934c-8f5bfad3b887 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:49.927362 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8731e6-13e6-42e6-934c-8f5bfad3b887 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:49.927643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8731e6-13e6-42e6-934c-8f5bfad3b887 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:49.927916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8731e6-13e6-42e6-934c-8f5bfad3b887 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:49.938598 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8731e6-13e6-42e6-934c-8f5bfad3b887 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:49.940069 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8731e6-13e6-42e6-934c-8f5bfad3b887 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:49.959564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1121/2258] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:50.093668 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd6a09-463e-4a34-a33b-e95216ee73f2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:50.103300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd6a09-463e-4a34-a33b-e95216ee73f2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:50.107004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4d53f-cbf5-4f9c-a199-a65df532e3e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:50.107657 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4d53f-cbf5-4f9c-a199-a65df532e3e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:50.107941 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4d53f-cbf5-4f9c-a199-a65df532e3e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:50.108217 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4d53f-cbf5-4f9c-a199-a65df532e3e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:50.188665 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4d53f-cbf5-4f9c-a199-a65df532e3e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:50.191058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4d53f-cbf5-4f9c-a199-a65df532e3e9 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:50.208882 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1138/2259] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:50.541520 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8731e6-13e6-42e6-934c-8f5bfad3b887 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:50.644896 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8731e6-13e6-42e6-934c-8f5bfad3b887 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:50.649707 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9977e92-b8d7-4187-b947-f63309437971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:50.651082 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9977e92-b8d7-4187-b947-f63309437971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:50.651364 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9977e92-b8d7-4187-b947-f63309437971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:50.652206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9977e92-b8d7-4187-b947-f63309437971 admin admin] PATH_INFO: `/v3/users/d98f386fceaa44ea80c5e0b88928d4ea` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:50.654168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9977e92-b8d7-4187-b947-f63309437971 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d98f386fceaa44ea80c5e0b88928d4ea)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:50.656347 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9977e92-b8d7-4187-b947-f63309437971 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:50.744244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d9977e92-b8d7-4187-b947-f63309437971 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd98f386fceaa44ea80c5e0b88928d4ea'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:50.745004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d9977e92-b8d7-4187-b947-f63309437971 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd98f386fceaa44ea80c5e0b88928d4ea'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:50.753770 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d9977e92-b8d7-4187-b947-f63309437971 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd98f386fceaa44ea80c5e0b88928d4ea'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:50.767197 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d9977e92-b8d7-4187-b947-f63309437971 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd98f386fceaa44ea80c5e0b88928d4ea'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:50.767927 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d9977e92-b8d7-4187-b947-f63309437971 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd98f386fceaa44ea80c5e0b88928d4ea'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:50.783493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1122/2260] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:01:50 2022] DELETE /identity/v3/users/d98f386fceaa44ea80c5e0b88928d4ea => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:50.788719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02461c-d8b1-4262-924e-05258ee543e5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:50.789036 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02461c-d8b1-4262-924e-05258ee543e5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:50.789392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02461c-d8b1-4262-924e-05258ee543e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:50.810323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-bb02461c-d8b1-4262-924e-05258ee543e5 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:50.811580 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bb02461c-d8b1-4262-924e-05258ee543e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:50.883489 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1139/2261] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:01:50.895086 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9977e92-b8d7-4187-b947-f63309437971 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:50.896525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d9977e92-b8d7-4187-b947-f63309437971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:50.916724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:51.090890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9977e92-b8d7-4187-b947-f63309437971 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:51.094837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa1330-7596-4858-b969-2accca00cc88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:51.095492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa1330-7596-4858-b969-2accca00cc88 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:51.095774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa1330-7596-4858-b969-2accca00cc88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:51.096045 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa1330-7596-4858-b969-2accca00cc88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:51.109344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa1330-7596-4858-b969-2accca00cc88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:51.112212 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa1330-7596-4858-b969-2accca00cc88 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:51.131360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1123/2262] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:51.910564 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02461c-d8b1-4262-924e-05258ee543e5 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:51.912743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bb02461c-d8b1-4262-924e-05258ee543e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:51.926104 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02461c-d8b1-4262-924e-05258ee543e5 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:51.934916 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:51.936086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:51.936438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:51.936785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35 admin admin] PATH_INFO: `/v3/projects/8e6a75170a0a4dea916f688ecc927b0a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:51.938040 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e6a75170a0a4dea916f688ecc927b0a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:51.940071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:51.977081 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:52.016046 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e6a75170a0a4dea916f688ecc927b0a', 'request_id': 'req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:52.025379 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e6a75170a0a4dea916f688ecc927b0a', 'request_id': 'req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:52.032477 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e6a75170a0a4dea916f688ecc927b0a', 'request_id': 'req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:52.044972 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35 admin admin] The token cache is being invalidate because project 8e6a75170a0a4dea916f688ecc927b0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:01:52.045265 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:52.045964 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:52.047539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1140/2263] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:01:51 2022] DELETE /identity/v3/projects/8e6a75170a0a4dea916f688ecc927b0a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:52.051529 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa1330-7596-4858-b969-2accca00cc88 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:52.062978 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa1330-7596-4858-b969-2accca00cc88 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:52.130053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc33012-3a38-463c-816d-84922e643fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:52.130696 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc33012-3a38-463c-816d-84922e643fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:52.131023 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc33012-3a38-463c-816d-84922e643fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:52.131319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc33012-3a38-463c-816d-84922e643fde admin admin] PATH_INFO: `/v3/users/dc50b516e27644a7910cd83a3632d657` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:52.134184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc33012-3a38-463c-816d-84922e643fde admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc50b516e27644a7910cd83a3632d657)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:52.136959 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc33012-3a38-463c-816d-84922e643fde admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:52.190049 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2fc33012-3a38-463c-816d-84922e643fde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc50b516e27644a7910cd83a3632d657'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:52.191112 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2fc33012-3a38-463c-816d-84922e643fde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc50b516e27644a7910cd83a3632d657'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:52.199291 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2fc33012-3a38-463c-816d-84922e643fde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc50b516e27644a7910cd83a3632d657'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:52.213856 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2fc33012-3a38-463c-816d-84922e643fde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc50b516e27644a7910cd83a3632d657'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:52.215225 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2fc33012-3a38-463c-816d-84922e643fde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc50b516e27644a7910cd83a3632d657'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:52.230199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1124/2264] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:01:52 2022] DELETE /identity/v3/users/dc50b516e27644a7910cd83a3632d657 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:52.580833 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:52.582416 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:52.596317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d9bd3-1289-4cfc-bb4e-bf0adcfa4a35 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:52.656067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37f02a-f625-488e-88bb-acffd747c853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:52.657270 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37f02a-f625-488e-88bb-acffd747c853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:52.657906 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37f02a-f625-488e-88bb-acffd747c853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:52.658213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37f02a-f625-488e-88bb-acffd747c853 admin admin] PATH_INFO: `/v3/projects/2689a520b57a432c893da8023605d4b0` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:52.659649 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37f02a-f625-488e-88bb-acffd747c853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2689a520b57a432c893da8023605d4b0)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:52.662142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37f02a-f625-488e-88bb-acffd747c853 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:52.728166 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6a37f02a-f625-488e-88bb-acffd747c853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2689a520b57a432c893da8023605d4b0', 'request_id': 'req-6a37f02a-f625-488e-88bb-acffd747c853'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:52.741165 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6a37f02a-f625-488e-88bb-acffd747c853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2689a520b57a432c893da8023605d4b0', 'request_id': 'req-6a37f02a-f625-488e-88bb-acffd747c853'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:52.749727 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6a37f02a-f625-488e-88bb-acffd747c853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2689a520b57a432c893da8023605d4b0', 'request_id': 'req-6a37f02a-f625-488e-88bb-acffd747c853'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:52.764304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6a37f02a-f625-488e-88bb-acffd747c853 admin admin] The token cache is being invalidate because project 2689a520b57a432c893da8023605d4b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:01:52.764671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6a37f02a-f625-488e-88bb-acffd747c853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:52.765467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6a37f02a-f625-488e-88bb-acffd747c853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:52.770204 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1141/2265] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:01:52 2022] DELETE /identity/v3/projects/2689a520b57a432c893da8023605d4b0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:52.772467 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d818429-86fc-44dd-b323-de155122119d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:52.772972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d818429-86fc-44dd-b323-de155122119d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:52.773255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d818429-86fc-44dd-b323-de155122119d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:52.794293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-3d818429-86fc-44dd-b323-de155122119d None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:52.795609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3d818429-86fc-44dd-b323-de155122119d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:52.828766 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:52.887404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1125/2266] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:52.897180 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1c6e7-1c57-41e9-9278-dd6075269612 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:52.897472 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1c6e7-1c57-41e9-9278-dd6075269612 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:52.897750 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1c6e7-1c57-41e9-9278-dd6075269612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:52.925002 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-77f1c6e7-1c57-41e9-9278-dd6075269612 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:52.926526 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-77f1c6e7-1c57-41e9-9278-dd6075269612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:52.940892 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1142/2267] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:52.949033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d818429-86fc-44dd-b323-de155122119d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:52.960600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d818429-86fc-44dd-b323-de155122119d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:52.967508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9815a-f4be-4248-82d6-2ffdd119c23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:52.968242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9815a-f4be-4248-82d6-2ffdd119c23a admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:52.968607 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9815a-f4be-4248-82d6-2ffdd119c23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:52.968862 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9815a-f4be-4248-82d6-2ffdd119c23a admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:52.969846 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9815a-f4be-4248-82d6-2ffdd119c23a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:01:52.970166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9815a-f4be-4248-82d6-2ffdd119c23a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:52.971970 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9815a-f4be-4248-82d6-2ffdd119c23a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:52.994830 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1126/2268] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:01:52.996848 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1c6e7-1c57-41e9-9278-dd6075269612 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:53.006060 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1c6e7-1c57-41e9-9278-dd6075269612 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:53.015439 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c19e5-a101-4436-ad8e-d6190998c2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:53.016437 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c19e5-a101-4436-ad8e-d6190998c2a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:53.016937 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c19e5-a101-4436-ad8e-d6190998c2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:53.017433 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c19e5-a101-4436-ad8e-d6190998c2a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:53.018692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c19e5-a101-4436-ad8e-d6190998c2a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:53.021463 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c19e5-a101-4436-ad8e-d6190998c2a4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:53.052099 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1143/2269] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:01:52 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:01:53.062514 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9815a-f4be-4248-82d6-2ffdd119c23a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:53.076893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9815a-f4be-4248-82d6-2ffdd119c23a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:53.148183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b760e5-da31-4b54-a6b5-dfcc8778625a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:53.148856 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b760e5-da31-4b54-a6b5-dfcc8778625a admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:53.149137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b760e5-da31-4b54-a6b5-dfcc8778625a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:53.149433 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b760e5-da31-4b54-a6b5-dfcc8778625a admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:53.150101 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b760e5-da31-4b54-a6b5-dfcc8778625a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:53.153125 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b760e5-da31-4b54-a6b5-dfcc8778625a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:53.178549 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1127/2270] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:01:53 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:53.182306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c19e5-a101-4436-ad8e-d6190998c2a4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:53.200917 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c19e5-a101-4436-ad8e-d6190998c2a4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:53.209102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50719e08-cc51-40a4-9e18-936cac8fb804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:53.209941 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50719e08-cc51-40a4-9e18-936cac8fb804 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:53.210301 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50719e08-cc51-40a4-9e18-936cac8fb804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:53.210604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50719e08-cc51-40a4-9e18-936cac8fb804 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:53.211586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50719e08-cc51-40a4-9e18-936cac8fb804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:53.213268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50719e08-cc51-40a4-9e18-936cac8fb804 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:53.220224 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:53.230880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1144/2271] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:53.231793 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b760e5-da31-4b54-a6b5-dfcc8778625a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:53.245076 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b760e5-da31-4b54-a6b5-dfcc8778625a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:53.250148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490935a-dc93-4c72-809a-275574cfbdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:53.250879 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490935a-dc93-4c72-809a-275574cfbdd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:53.251280 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490935a-dc93-4c72-809a-275574cfbdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:53.251634 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490935a-dc93-4c72-809a-275574cfbdd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:53.252448 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8490935a-dc93-4c72-809a-275574cfbdd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:53.254307 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8490935a-dc93-4c72-809a-275574cfbdd0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:53.267671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1128/2272] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:01:53.271468 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50719e08-cc51-40a4-9e18-936cac8fb804 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:53.282748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50719e08-cc51-40a4-9e18-936cac8fb804 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:53.287295 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e62865-3743-44cc-94c2-e3fcf95e4af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:53.288177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e62865-3743-44cc-94c2-e3fcf95e4af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:53.288459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e62865-3743-44cc-94c2-e3fcf95e4af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:53.288734 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e62865-3743-44cc-94c2-e3fcf95e4af3 admin admin] PATH_INFO: `/v3/projects/5c82ec7de8174fc6874d6e63093c93fe/users/6ef0bea7b27a4923baac6df40fe34cf8/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:53.304518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e62865-3743-44cc-94c2-e3fcf95e4af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c82ec7de8174fc6874d6e63093c93fe, user_id=6ef0bea7b27a4923baac6df40fe34cf8, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:53.307229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e62865-3743-44cc-94c2-e3fcf95e4af3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:53.320513 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1145/2273] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:53 2022] PUT /identity/v3/projects/5c82ec7de8174fc6874d6e63093c93fe/users/6ef0bea7b27a4923baac6df40fe34cf8/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:53.332780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b55ea2-b768-4704-b69f-4fd66e3a3e71 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:53.333083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b55ea2-b768-4704-b69f-4fd66e3a3e71 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:53.333387 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b55ea2-b768-4704-b69f-4fd66e3a3e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:53.363718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-87b55ea2-b768-4704-b69f-4fd66e3a3e71 None None] MFA Rules not processed for user `6ef0bea7b27a4923baac6df40fe34cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:53.364939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-87b55ea2-b768-4704-b69f-4fd66e3a3e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:53.434343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1129/2274] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:53.446602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e62865-3743-44cc-94c2-e3fcf95e4af3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:53.448930 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-21e62865-3743-44cc-94c2-e3fcf95e4af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:53.508995 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e62865-3743-44cc-94c2-e3fcf95e4af3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:53.513311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f09c8-b8c5-4995-8dcd-ec1ab0f8fa82 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8563a25d16134d639a2a8849976fed9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:53.513957 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f09c8-b8c5-4995-8dcd-ec1ab0f8fa82 service glance] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:53.514241 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f09c8-b8c5-4995-8dcd-ec1ab0f8fa82 service glance] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:53.514522 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f09c8-b8c5-4995-8dcd-ec1ab0f8fa82 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:53.523285 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f09c8-b8c5-4995-8dcd-ec1ab0f8fa82 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:53.524742 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f09c8-b8c5-4995-8dcd-ec1ab0f8fa82 service glance] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:53.534696 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1146/2275] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:56.568891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b55ea2-b768-4704-b69f-4fd66e3a3e71 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:56.570512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-87b55ea2-b768-4704-b69f-4fd66e3a3e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:56.602014 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:56.641144 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b55ea2-b768-4704-b69f-4fd66e3a3e71 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:56.645740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6114985-f67c-4366-9fef-3cce12dda00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:56.646639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6114985-f67c-4366-9fef-3cce12dda00e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:56.646918 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6114985-f67c-4366-9fef-3cce12dda00e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:56.647264 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6114985-f67c-4366-9fef-3cce12dda00e admin admin] PATH_INFO: `/v3/users/292561b2e6d6408c928c6302b63ad9c7` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:56.648560 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6114985-f67c-4366-9fef-3cce12dda00e admin admin] RBAC: Authorizing `identity:delete_user(user_id=292561b2e6d6408c928c6302b63ad9c7)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:56.650939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6114985-f67c-4366-9fef-3cce12dda00e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:56.703873 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e6114985-f67c-4366-9fef-3cce12dda00e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '292561b2e6d6408c928c6302b63ad9c7'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:56.704553 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e6114985-f67c-4366-9fef-3cce12dda00e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '292561b2e6d6408c928c6302b63ad9c7'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:56.711536 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e6114985-f67c-4366-9fef-3cce12dda00e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '292561b2e6d6408c928c6302b63ad9c7'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:56.725286 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e6114985-f67c-4366-9fef-3cce12dda00e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '292561b2e6d6408c928c6302b63ad9c7'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:56.726039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e6114985-f67c-4366-9fef-3cce12dda00e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '292561b2e6d6408c928c6302b63ad9c7'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:56.740461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1130/2276] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:01:56 2022] DELETE /identity/v3/users/292561b2e6d6408c928c6302b63ad9c7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:56.745272 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39286363-4c80-4368-9256-81fdf4c970f8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:56.745971 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39286363-4c80-4368-9256-81fdf4c970f8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:56.745971 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39286363-4c80-4368-9256-81fdf4c970f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:56.764881 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-39286363-4c80-4368-9256-81fdf4c970f8 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:56.766069 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-39286363-4c80-4368-9256-81fdf4c970f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:56.845301 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1147/2277] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:01:56.855742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6114985-f67c-4366-9fef-3cce12dda00e admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:56.857184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e6114985-f67c-4366-9fef-3cce12dda00e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:56.927929 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6114985-f67c-4366-9fef-3cce12dda00e admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:56.931686 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23317f-ab73-4bba-b6a0-3c8c021f5bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:56.932338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23317f-ab73-4bba-b6a0-3c8c021f5bce service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:56.932694 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23317f-ab73-4bba-b6a0-3c8c021f5bce service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:56.932975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23317f-ab73-4bba-b6a0-3c8c021f5bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:56.943357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23317f-ab73-4bba-b6a0-3c8c021f5bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:56.944815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23317f-ab73-4bba-b6a0-3c8c021f5bce service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:56.955959 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1131/2278] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:57.251253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39286363-4c80-4368-9256-81fdf4c970f8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:57.252900 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-39286363-4c80-4368-9256-81fdf4c970f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:57.278077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39286363-4c80-4368-9256-81fdf4c970f8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:57.284467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2566d9-20bb-4296-857a-78ae8f21c3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:57.285715 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2566d9-20bb-4296-857a-78ae8f21c3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:57.285715 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2566d9-20bb-4296-857a-78ae8f21c3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:57.285928 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2566d9-20bb-4296-857a-78ae8f21c3d5 admin admin] PATH_INFO: `/v3/projects/b585ac2789ae4143af8a40926b14101c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:57.287134 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2566d9-20bb-4296-857a-78ae8f21c3d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b585ac2789ae4143af8a40926b14101c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:57.289060 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2566d9-20bb-4296-857a-78ae8f21c3d5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:57.355461 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7c2566d9-20bb-4296-857a-78ae8f21c3d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b585ac2789ae4143af8a40926b14101c', 'request_id': 'req-7c2566d9-20bb-4296-857a-78ae8f21c3d5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:57.364058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7c2566d9-20bb-4296-857a-78ae8f21c3d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b585ac2789ae4143af8a40926b14101c', 'request_id': 'req-7c2566d9-20bb-4296-857a-78ae8f21c3d5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:57.371565 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7c2566d9-20bb-4296-857a-78ae8f21c3d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b585ac2789ae4143af8a40926b14101c', 'request_id': 'req-7c2566d9-20bb-4296-857a-78ae8f21c3d5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:57.390289 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7c2566d9-20bb-4296-857a-78ae8f21c3d5 admin admin] The token cache is being invalidate because project b585ac2789ae4143af8a40926b14101c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:01:57.390598 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7c2566d9-20bb-4296-857a-78ae8f21c3d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:57.391332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7c2566d9-20bb-4296-857a-78ae8f21c3d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:57.392999 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1148/2279] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:01:57 2022] DELETE /identity/v3/projects/b585ac2789ae4143af8a40926b14101c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:57.398264 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b5418-3245-4409-8ed9-7b5a1bc65fd3 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:57.398550 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b5418-3245-4409-8ed9-7b5a1bc65fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:57.398865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b5418-3245-4409-8ed9-7b5a1bc65fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:57.419727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ab3b5418-3245-4409-8ed9-7b5a1bc65fd3 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:57.421028 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ab3b5418-3245-4409-8ed9-7b5a1bc65fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:57.486650 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:57.515727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1132/2280] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:57.523444 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603c455-21fb-4bc5-873c-3a156093ddae None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:57.523806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603c455-21fb-4bc5-873c-3a156093ddae None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:57.524179 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603c455-21fb-4bc5-873c-3a156093ddae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:57.531960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:57.546996 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a603c455-21fb-4bc5-873c-3a156093ddae None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:57.548324 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a603c455-21fb-4bc5-873c-3a156093ddae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:57.556225 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1149/2281] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:57.560848 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b5418-3245-4409-8ed9-7b5a1bc65fd3 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:57.571867 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b5418-3245-4409-8ed9-7b5a1bc65fd3 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:57.577141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec04b12-4d96-4713-a875-1811b541d950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:57.578099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec04b12-4d96-4713-a875-1811b541d950 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:57.578483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec04b12-4d96-4713-a875-1811b541d950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:57.578891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec04b12-4d96-4713-a875-1811b541d950 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:57.579866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec04b12-4d96-4713-a875-1811b541d950 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:01:57.580283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec04b12-4d96-4713-a875-1811b541d950 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:57.582356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec04b12-4d96-4713-a875-1811b541d950 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:57.596927 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1133/2282] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:01:57.601845 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603c455-21fb-4bc5-873c-3a156093ddae None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:57.612539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603c455-21fb-4bc5-873c-3a156093ddae None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:57.617634 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f826f-020f-4717-baa2-a214b985cb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:57.618397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f826f-020f-4717-baa2-a214b985cb90 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:57.618801 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f826f-020f-4717-baa2-a214b985cb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:57.619088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f826f-020f-4717-baa2-a214b985cb90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:57.619800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f826f-020f-4717-baa2-a214b985cb90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:57.622449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f826f-020f-4717-baa2-a214b985cb90 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:57.650187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1150/2283] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:01:57 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:57.655688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec04b12-4d96-4713-a875-1811b541d950 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:57.666140 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec04b12-4d96-4713-a875-1811b541d950 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:57.728796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c44e35-d3f4-439f-ab6e-b604a5f0f444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:57.730075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c44e35-d3f4-439f-ab6e-b604a5f0f444 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:57.730703 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c44e35-d3f4-439f-ab6e-b604a5f0f444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:57.730703 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c44e35-d3f4-439f-ab6e-b604a5f0f444 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:57.731436 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c44e35-d3f4-439f-ab6e-b604a5f0f444 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:57.733455 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c44e35-d3f4-439f-ab6e-b604a5f0f444 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:57.753731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1134/2284] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:01:57 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:01:57.757790 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f826f-020f-4717-baa2-a214b985cb90 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:57.769674 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f826f-020f-4717-baa2-a214b985cb90 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:57.774307 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7d216-3669-46be-8f36-7e0e7c1f25e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:57.775034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7d216-3669-46be-8f36-7e0e7c1f25e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:57.775323 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7d216-3669-46be-8f36-7e0e7c1f25e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:57.775596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7d216-3669-46be-8f36-7e0e7c1f25e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:57.777073 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7d216-3669-46be-8f36-7e0e7c1f25e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:57.778703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7d216-3669-46be-8f36-7e0e7c1f25e7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:57.790475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1151/2285] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:57.795544 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c44e35-d3f4-439f-ab6e-b604a5f0f444 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:57.807369 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c44e35-d3f4-439f-ab6e-b604a5f0f444 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:57.813635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6b0cc-c66e-462e-8e63-39369966abb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:57.814301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6b0cc-c66e-462e-8e63-39369966abb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:57.814608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6b0cc-c66e-462e-8e63-39369966abb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:57.814893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6b0cc-c66e-462e-8e63-39369966abb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:57.815615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6b0cc-c66e-462e-8e63-39369966abb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:57.817221 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6b0cc-c66e-462e-8e63-39369966abb6 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:57.833506 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1135/2286] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:57.837833 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7d216-3669-46be-8f36-7e0e7c1f25e7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:57.847996 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7d216-3669-46be-8f36-7e0e7c1f25e7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:57.852637 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9439e-163c-462f-a9ea-3002cf79f629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:57.853561 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9439e-163c-462f-a9ea-3002cf79f629 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:57.853844 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9439e-163c-462f-a9ea-3002cf79f629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:57.854120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9439e-163c-462f-a9ea-3002cf79f629 admin admin] PATH_INFO: `/v3/projects/bae3331bfc29456c94e309c51cb82a9f/users/7e66f41aa3194667bd5970337fd41cfa/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:57.869626 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9439e-163c-462f-a9ea-3002cf79f629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bae3331bfc29456c94e309c51cb82a9f, user_id=7e66f41aa3194667bd5970337fd41cfa, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:57.872307 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9439e-163c-462f-a9ea-3002cf79f629 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:57.887389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1152/2287] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:57 2022] PUT /identity/v3/projects/bae3331bfc29456c94e309c51cb82a9f/users/7e66f41aa3194667bd5970337fd41cfa/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:57.897885 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2d170-9123-4455-a93b-a3cc6ed6dc6b None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:57.898176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2d170-9123-4455-a93b-a3cc6ed6dc6b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:57.898459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2d170-9123-4455-a93b-a3cc6ed6dc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:57.925643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-64e2d170-9123-4455-a93b-a3cc6ed6dc6b None None] MFA Rules not processed for user `7e66f41aa3194667bd5970337fd41cfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:57.926899 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-64e2d170-9123-4455-a93b-a3cc6ed6dc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:57.998221 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1136/2288] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:58.008942 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9439e-163c-462f-a9ea-3002cf79f629 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:58.010909 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-10c9439e-163c-462f-a9ea-3002cf79f629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:58.073035 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9439e-163c-462f-a9ea-3002cf79f629 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:58.077215 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6cedf-6152-4584-9158-8721364713c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:58.077896 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6cedf-6152-4584-9158-8721364713c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:58.078184 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6cedf-6152-4584-9158-8721364713c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:58.078462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6cedf-6152-4584-9158-8721364713c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:58.087216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6cedf-6152-4584-9158-8721364713c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:58.088639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6cedf-6152-4584-9158-8721364713c0 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:58.098983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1153/2289] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:59.497184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fd663-de92-4ece-8895-14f6846a83c4 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:59.497662 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fd663-de92-4ece-8895-14f6846a83c4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:59.497897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fd663-de92-4ece-8895-14f6846a83c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:59.516702 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-349fd663-de92-4ece-8895-14f6846a83c4 None None] MFA Rules not processed for user `7e66f41aa3194667bd5970337fd41cfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:59.517961 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-349fd663-de92-4ece-8895-14f6846a83c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:59.527961 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1137/2290] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:01:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:05.377916 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be124dd5-278d-4b6a-a7a5-0dfa2172ab22 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:05.378382 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be124dd5-278d-4b6a-a7a5-0dfa2172ab22 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:05.378533 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be124dd5-278d-4b6a-a7a5-0dfa2172ab22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:05.400171 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-be124dd5-278d-4b6a-a7a5-0dfa2172ab22 None None] MFA Rules not processed for user `02c826fc0baa480393df4c884c9e35dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:05.403028 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-be124dd5-278d-4b6a-a7a5-0dfa2172ab22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:05.473409 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1154/2291] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Jun 16 19:02:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:05.487459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fd663-de92-4ece-8895-14f6846a83c4 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:05.489431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-349fd663-de92-4ece-8895-14f6846a83c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:05.536394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:05.576937 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fd663-de92-4ece-8895-14f6846a83c4 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:05.582200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43067258-48f8-4eca-8595-c17e01721c46 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b22fe0ab0354385924923d0d443acae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:05.582894 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43067258-48f8-4eca-8595-c17e01721c46 service swift] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:05.583180 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43067258-48f8-4eca-8595-c17e01721c46 service swift] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:05.583460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43067258-48f8-4eca-8595-c17e01721c46 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:05.592925 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43067258-48f8-4eca-8595-c17e01721c46 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:05.594842 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43067258-48f8-4eca-8595-c17e01721c46 service swift] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:05.605566 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1138/2292] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Jun 16 19:02:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:02:11.997141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be124dd5-278d-4b6a-a7a5-0dfa2172ab22 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:12.000918 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-be124dd5-278d-4b6a-a7a5-0dfa2172ab22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:12.056497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:12.125747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be124dd5-278d-4b6a-a7a5-0dfa2172ab22 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:12.131953 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbecbd-7159-4020-a569-062c744cca9f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '880d0299c38c4eb897c669a512e599be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:12.132745 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbecbd-7159-4020-a569-062c744cca9f service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:12.133099 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbecbd-7159-4020-a569-062c744cca9f service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:12.133464 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbecbd-7159-4020-a569-062c744cca9f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:12.137044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7bbbecbd-7159-4020-a569-062c744cca9f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:12.162316 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbbecbd-7159-4020-a569-062c744cca9f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:12.164608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbbecbd-7159-4020-a569-062c744cca9f service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:12.179367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1155/2293] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:15.600660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=15 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:19.772429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43067258-48f8-4eca-8595-c17e01721c46 service swift] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:19.773999 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-43067258-48f8-4eca-8595-c17e01721c46 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:19.835765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43067258-48f8-4eca-8595-c17e01721c46 service swift] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:19.840170 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5855447-fe2f-454a-ae72-82eacc64d5e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:19.840810 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855447-fe2f-454a-ae72-82eacc64d5e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:19.841092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855447-fe2f-454a-ae72-82eacc64d5e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:19.841366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855447-fe2f-454a-ae72-82eacc64d5e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:19.843177 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d5855447-fe2f-454a-ae72-82eacc64d5e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:19.905920 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5855447-fe2f-454a-ae72-82eacc64d5e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:19.907383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5855447-fe2f-454a-ae72-82eacc64d5e8 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:19.917974 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1139/2294] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:21.142945 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9122bc5-c0c4-4ce4-b13c-30da757a5f99 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:21.143548 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9122bc5-c0c4-4ce4-b13c-30da757a5f99 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:21.143654 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9122bc5-c0c4-4ce4-b13c-30da757a5f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:21.162304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f9122bc5-c0c4-4ce4-b13c-30da757a5f99 None None] MFA Rules not processed for user `922a970a740444a2959c6aa1d0921ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:21.163669 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f9122bc5-c0c4-4ce4-b13c-30da757a5f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:21.224478 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1156/2295] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:24.367954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5855447-fe2f-454a-ae72-82eacc64d5e8 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:24.369565 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d5855447-fe2f-454a-ae72-82eacc64d5e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:24.541769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5855447-fe2f-454a-ae72-82eacc64d5e8 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:24.552944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7a28e-cab1-473a-926b-eec4275fc4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:24.554011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7a28e-cab1-473a-926b-eec4275fc4f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:24.554373 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7a28e-cab1-473a-926b-eec4275fc4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:24.554740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7a28e-cab1-473a-926b-eec4275fc4f4 admin admin] PATH_INFO: `/v3/users/6ef0bea7b27a4923baac6df40fe34cf8` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:24.560596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7a28e-cab1-473a-926b-eec4275fc4f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ef0bea7b27a4923baac6df40fe34cf8)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:24.564509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7a28e-cab1-473a-926b-eec4275fc4f4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:24.670845 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e8a7a28e-cab1-473a-926b-eec4275fc4f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ef0bea7b27a4923baac6df40fe34cf8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:24.671855 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e8a7a28e-cab1-473a-926b-eec4275fc4f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ef0bea7b27a4923baac6df40fe34cf8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:24.677268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:24.684424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e8a7a28e-cab1-473a-926b-eec4275fc4f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ef0bea7b27a4923baac6df40fe34cf8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:24.720512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e8a7a28e-cab1-473a-926b-eec4275fc4f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ef0bea7b27a4923baac6df40fe34cf8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:24.721542 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e8a7a28e-cab1-473a-926b-eec4275fc4f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ef0bea7b27a4923baac6df40fe34cf8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:24.756684 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1140/2296] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:02:24 2022] DELETE /identity/v3/users/6ef0bea7b27a4923baac6df40fe34cf8 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:24.761932 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd6bc6-c7e8-4738-909f-eb77583b68b8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:24.762802 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd6bc6-c7e8-4738-909f-eb77583b68b8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:24.763227 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd6bc6-c7e8-4738-909f-eb77583b68b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:24.794017 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d4bd6bc6-c7e8-4738-909f-eb77583b68b8 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:24.796053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d4bd6bc6-c7e8-4738-909f-eb77583b68b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:24.884502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1157/2297] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:24.896095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7a28e-cab1-473a-926b-eec4275fc4f4 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:24.897813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e8a7a28e-cab1-473a-926b-eec4275fc4f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:24.987008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7a28e-cab1-473a-926b-eec4275fc4f4 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:24.991706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cceed0-905d-410a-9308-635ad033208c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:24.992482 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cceed0-905d-410a-9308-635ad033208c service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:24.992822 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cceed0-905d-410a-9308-635ad033208c service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:24.993346 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cceed0-905d-410a-9308-635ad033208c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:25.012707 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cceed0-905d-410a-9308-635ad033208c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:25.014261 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cceed0-905d-410a-9308-635ad033208c service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:25.026891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1141/2298] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:25.538081 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd6bc6-c7e8-4738-909f-eb77583b68b8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:25.539928 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d4bd6bc6-c7e8-4738-909f-eb77583b68b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:25.553812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd6bc6-c7e8-4738-909f-eb77583b68b8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:25.559007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983241a-640c-44a5-8bc6-8f2eb4935873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:25.560065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983241a-640c-44a5-8bc6-8f2eb4935873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:25.560345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983241a-640c-44a5-8bc6-8f2eb4935873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:25.560617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983241a-640c-44a5-8bc6-8f2eb4935873 admin admin] PATH_INFO: `/v3/projects/5c82ec7de8174fc6874d6e63093c93fe` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:25.562019 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983241a-640c-44a5-8bc6-8f2eb4935873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c82ec7de8174fc6874d6e63093c93fe)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:25.564140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983241a-640c-44a5-8bc6-8f2eb4935873 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:25.617665 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8983241a-640c-44a5-8bc6-8f2eb4935873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c82ec7de8174fc6874d6e63093c93fe', 'request_id': 'req-8983241a-640c-44a5-8bc6-8f2eb4935873'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:25.619717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:25.625463 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8983241a-640c-44a5-8bc6-8f2eb4935873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c82ec7de8174fc6874d6e63093c93fe', 'request_id': 'req-8983241a-640c-44a5-8bc6-8f2eb4935873'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:25.632804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8983241a-640c-44a5-8bc6-8f2eb4935873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c82ec7de8174fc6874d6e63093c93fe', 'request_id': 'req-8983241a-640c-44a5-8bc6-8f2eb4935873'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:25.649551 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8983241a-640c-44a5-8bc6-8f2eb4935873 admin admin] The token cache is being invalidate because project 5c82ec7de8174fc6874d6e63093c93fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:02:25.649859 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8983241a-640c-44a5-8bc6-8f2eb4935873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:25.651315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8983241a-640c-44a5-8bc6-8f2eb4935873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:25.652932 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1158/2299] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:02:25 2022] DELETE /identity/v3/projects/5c82ec7de8174fc6874d6e63093c93fe => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:25.657811 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c808b6f-e090-473f-b9c5-54f46a11090b None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:25.658278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c808b6f-e090-473f-b9c5-54f46a11090b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:25.658477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c808b6f-e090-473f-b9c5-54f46a11090b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:25.677223 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-9c808b6f-e090-473f-b9c5-54f46a11090b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:25.678471 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9c808b6f-e090-473f-b9c5-54f46a11090b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:25.748896 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1142/2300] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:02:25.750553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3b768-e6d6-40d2-af99-c966c68f08c0 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:25.751044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3b768-e6d6-40d2-af99-c966c68f08c0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:25.751486 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3b768-e6d6-40d2-af99-c966c68f08c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:25.771990 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-77c3b768-e6d6-40d2-af99-c966c68f08c0 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:25.773177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-77c3b768-e6d6-40d2-af99-c966c68f08c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:25.785703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1159/2301] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:02:25.787094 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c808b6f-e090-473f-b9c5-54f46a11090b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:25.809706 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c808b6f-e090-473f-b9c5-54f46a11090b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:25.815243 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfb6d1-c6fa-4004-8ec9-c16d25f6f582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:25.815986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfb6d1-c6fa-4004-8ec9-c16d25f6f582 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:25.816338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfb6d1-c6fa-4004-8ec9-c16d25f6f582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:25.817219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfb6d1-c6fa-4004-8ec9-c16d25f6f582 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:25.817492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfb6d1-c6fa-4004-8ec9-c16d25f6f582 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:02:25.817843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfb6d1-c6fa-4004-8ec9-c16d25f6f582 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:25.819551 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfb6d1-c6fa-4004-8ec9-c16d25f6f582 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:25.842609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1143/2302] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:02:25.844406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3b768-e6d6-40d2-af99-c966c68f08c0 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:25.856039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3b768-e6d6-40d2-af99-c966c68f08c0 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:25.860862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1c0b7-8f54-4fd2-b046-fb1bba22ec9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:25.861533 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1c0b7-8f54-4fd2-b046-fb1bba22ec9c admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:25.861822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1c0b7-8f54-4fd2-b046-fb1bba22ec9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:25.862103 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1c0b7-8f54-4fd2-b046-fb1bba22ec9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:25.862834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1c0b7-8f54-4fd2-b046-fb1bba22ec9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:25.864753 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1c0b7-8f54-4fd2-b046-fb1bba22ec9c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:25.897095 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1160/2303] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:02:25 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:25.902786 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfb6d1-c6fa-4004-8ec9-c16d25f6f582 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:25.913295 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfb6d1-c6fa-4004-8ec9-c16d25f6f582 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:25.975733 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3139c2-c1e3-4876-b891-15da2b6af669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:25.976397 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3139c2-c1e3-4876-b891-15da2b6af669 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:25.976678 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3139c2-c1e3-4876-b891-15da2b6af669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:25.976945 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3139c2-c1e3-4876-b891-15da2b6af669 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:25.977642 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3139c2-c1e3-4876-b891-15da2b6af669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:25.979729 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3139c2-c1e3-4876-b891-15da2b6af669 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:25.986818 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:26.003007 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1144/2304] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:02:25 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:26.007142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1c0b7-8f54-4fd2-b046-fb1bba22ec9c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:26.018111 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1c0b7-8f54-4fd2-b046-fb1bba22ec9c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:26.022872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca52102-97e5-4755-b641-31610db681e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:26.023527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca52102-97e5-4755-b641-31610db681e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:26.023811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca52102-97e5-4755-b641-31610db681e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:26.024086 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca52102-97e5-4755-b641-31610db681e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:26.024804 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca52102-97e5-4755-b641-31610db681e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:26.027540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca52102-97e5-4755-b641-31610db681e8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:26.038790 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1161/2305] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:26.043519 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3139c2-c1e3-4876-b891-15da2b6af669 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:26.054615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3139c2-c1e3-4876-b891-15da2b6af669 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:26.059999 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d25a8-ca37-4a71-af09-5ddf1882d963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:26.060690 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d25a8-ca37-4a71-af09-5ddf1882d963 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:26.060970 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d25a8-ca37-4a71-af09-5ddf1882d963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:26.061271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d25a8-ca37-4a71-af09-5ddf1882d963 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:26.061990 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d25a8-ca37-4a71-af09-5ddf1882d963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:26.063654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d25a8-ca37-4a71-af09-5ddf1882d963 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:26.075523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1145/2306] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:26.079819 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca52102-97e5-4755-b641-31610db681e8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:26.090889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca52102-97e5-4755-b641-31610db681e8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:26.096229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c104fd6-0063-471c-8084-4c5bdb16b0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:26.097150 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c104fd6-0063-471c-8084-4c5bdb16b0d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:26.097434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c104fd6-0063-471c-8084-4c5bdb16b0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:26.097789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c104fd6-0063-471c-8084-4c5bdb16b0d9 admin admin] PATH_INFO: `/v3/projects/e682d32da6f24367b13bad53c448f0a0/users/e4f2d119de384f95a2055301313a9d07/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:26.114132 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c104fd6-0063-471c-8084-4c5bdb16b0d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e682d32da6f24367b13bad53c448f0a0, user_id=e4f2d119de384f95a2055301313a9d07, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:26.116956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c104fd6-0063-471c-8084-4c5bdb16b0d9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:26.132328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1162/2307] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:02:26 2022] PUT /identity/v3/projects/e682d32da6f24367b13bad53c448f0a0/users/e4f2d119de384f95a2055301313a9d07/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:26.144321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c3cc8-7b26-4f35-9411-c790d13c5fa5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:26.144652 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c3cc8-7b26-4f35-9411-c790d13c5fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:26.145023 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c3cc8-7b26-4f35-9411-c790d13c5fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:26.176208 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-3b6c3cc8-7b26-4f35-9411-c790d13c5fa5 None None] MFA Rules not processed for user `e4f2d119de384f95a2055301313a9d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:26.177410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3b6c3cc8-7b26-4f35-9411-c790d13c5fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:26.250555 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1146/2308] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:26.263676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c104fd6-0063-471c-8084-4c5bdb16b0d9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:26.265503 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3c104fd6-0063-471c-8084-4c5bdb16b0d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:26.325829 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c104fd6-0063-471c-8084-4c5bdb16b0d9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:26.329539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a772e62-2cf5-4e7c-93df-6e7aba6e57ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:26.330188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a772e62-2cf5-4e7c-93df-6e7aba6e57ac service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:26.330468 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a772e62-2cf5-4e7c-93df-6e7aba6e57ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:26.330807 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a772e62-2cf5-4e7c-93df-6e7aba6e57ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:26.340239 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a772e62-2cf5-4e7c-93df-6e7aba6e57ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:26.341663 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a772e62-2cf5-4e7c-93df-6e7aba6e57ac service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:26.350607 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1163/2309] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:02:26.559216 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb30ea-1fe4-4558-8635-d79d2bea105d None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:26.559678 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb30ea-1fe4-4558-8635-d79d2bea105d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:26.559961 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb30ea-1fe4-4558-8635-d79d2bea105d None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:26.561680 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1147/2310] 10.222.0.70 () {54 vars in 921 bytes} [Thu Jun 16 19:02:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:02:26.569431 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf2067-199b-4122-a6b4-ca9a107d23f4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:26.569916 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf2067-199b-4122-a6b4-ca9a107d23f4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:26.570367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf2067-199b-4122-a6b4-ca9a107d23f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:26.646253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-bedf2067-199b-4122-a6b4-ca9a107d23f4 None None] MFA Rules not processed for user `8342da838fe4462b9de2817ad3ac4ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:26.647740 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bedf2067-199b-4122-a6b4-ca9a107d23f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:26.714547 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:26.766174 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1164/2311] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Jun 16 19:02:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:26.779111 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb30ea-1fe4-4558-8635-d79d2bea105d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:26.780889 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-83eb30ea-1fe4-4558-8635-d79d2bea105d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:26.870201 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb30ea-1fe4-4558-8635-d79d2bea105d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:26.874956 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5a3dd-ac6b-410a-9838-4a901b9432eb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8563a25d16134d639a2a8849976fed9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:26.874956 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5a3dd-ac6b-410a-9838-4a901b9432eb service glance] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:26.874956 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5a3dd-ac6b-410a-9838-4a901b9432eb service glance] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:26.875288 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5a3dd-ac6b-410a-9838-4a901b9432eb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:26.886034 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5a3dd-ac6b-410a-9838-4a901b9432eb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:26.887689 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5a3dd-ac6b-410a-9838-4a901b9432eb service glance] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:26.898762 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1148/2312] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:27.000420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43c943-83eb-45ec-9142-2539af597cd0 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:27.000943 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43c943-83eb-45ec-9142-2539af597cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:27.001055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43c943-83eb-45ec-9142-2539af597cd0 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:27.002611 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1165/2313] 10.222.0.70 () {54 vars in 921 bytes} [Thu Jun 16 19:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:02:27.008929 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29312aa3-ffb1-4b02-9dd2-617b4533d94e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:27.009298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29312aa3-ffb1-4b02-9dd2-617b4533d94e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:27.010276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29312aa3-ffb1-4b02-9dd2-617b4533d94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:27.031460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-29312aa3-ffb1-4b02-9dd2-617b4533d94e None None] MFA Rules not processed for user `02a83f6e7aa448d6984a78914c1748cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:27.032698 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-29312aa3-ffb1-4b02-9dd2-617b4533d94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:27.080262 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:27.088495 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1149/2314] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Jun 16 19:02:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:27.099933 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43c943-83eb-45ec-9142-2539af597cd0 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:27.101603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0e43c943-83eb-45ec-9142-2539af597cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:27.115964 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43c943-83eb-45ec-9142-2539af597cd0 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:27.120546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c3402-39f0-4d3a-b3aa-6ab0d6199901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:27.121180 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c3402-39f0-4d3a-b3aa-6ab0d6199901 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:27.121465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c3402-39f0-4d3a-b3aa-6ab0d6199901 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:27.121744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c3402-39f0-4d3a-b3aa-6ab0d6199901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:27.129231 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c3402-39f0-4d3a-b3aa-6ab0d6199901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:27.130805 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c3402-39f0-4d3a-b3aa-6ab0d6199901 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:27.139237 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1166/2315] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:27.225459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54efb7a7-e788-4bfd-9588-80b2b7f23d1b None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:27.225751 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54efb7a7-e788-4bfd-9588-80b2b7f23d1b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:27.226031 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54efb7a7-e788-4bfd-9588-80b2b7f23d1b None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:27.227440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1150/2316] 10.222.0.70 () {54 vars in 921 bytes} [Thu Jun 16 19:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:02:27.234749 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83e2c8-8d2c-475a-af4d-0b697d70ca50 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:27.235046 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83e2c8-8d2c-475a-af4d-0b697d70ca50 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:27.235328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83e2c8-8d2c-475a-af4d-0b697d70ca50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:27.256183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ab83e2c8-8d2c-475a-af4d-0b697d70ca50 None None] MFA Rules not processed for user `8342da838fe4462b9de2817ad3ac4ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:27.257943 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ab83e2c8-8d2c-475a-af4d-0b697d70ca50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:27.267668 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1167/2317] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Jun 16 19:02:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:27.278504 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54efb7a7-e788-4bfd-9588-80b2b7f23d1b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:27.280257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-54efb7a7-e788-4bfd-9588-80b2b7f23d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:27.295588 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54efb7a7-e788-4bfd-9588-80b2b7f23d1b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:27.300741 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130efb1-88b1-400f-8998-8bb07f8050d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:27.301389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130efb1-88b1-400f-8998-8bb07f8050d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:27.301673 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130efb1-88b1-400f-8998-8bb07f8050d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:27.302066 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130efb1-88b1-400f-8998-8bb07f8050d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:27.313655 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130efb1-88b1-400f-8998-8bb07f8050d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:27.315135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130efb1-88b1-400f-8998-8bb07f8050d4 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:27.327373 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1151/2318] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:29.943193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83e2c8-8d2c-475a-af4d-0b697d70ca50 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:30.039455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83e2c8-8d2c-475a-af4d-0b697d70ca50 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:30.047652 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144b576-09af-47cf-98bd-a459e00d353a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:30.048136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144b576-09af-47cf-98bd-a459e00d353a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:30.048426 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144b576-09af-47cf-98bd-a459e00d353a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:30.048707 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144b576-09af-47cf-98bd-a459e00d353a admin admin] PATH_INFO: `/v3/users/e4f2d119de384f95a2055301313a9d07` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:30.049962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144b576-09af-47cf-98bd-a459e00d353a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4f2d119de384f95a2055301313a9d07)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:30.052120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144b576-09af-47cf-98bd-a459e00d353a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:30.122499 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a144b576-09af-47cf-98bd-a459e00d353a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4f2d119de384f95a2055301313a9d07'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:30.123120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a144b576-09af-47cf-98bd-a459e00d353a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4f2d119de384f95a2055301313a9d07'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:30.123720 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a144b576-09af-47cf-98bd-a459e00d353a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4f2d119de384f95a2055301313a9d07'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:30.134603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a144b576-09af-47cf-98bd-a459e00d353a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4f2d119de384f95a2055301313a9d07'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:30.145807 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a144b576-09af-47cf-98bd-a459e00d353a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4f2d119de384f95a2055301313a9d07'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:30.169191 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1168/2319] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:02:29 2022] DELETE /identity/v3/users/e4f2d119de384f95a2055301313a9d07 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:30.173775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f4136-1001-4fd2-b1ea-7d02f1115b82 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:30.174330 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f4136-1001-4fd2-b1ea-7d02f1115b82 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:30.174330 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f4136-1001-4fd2-b1ea-7d02f1115b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:30.203420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c99f4136-1001-4fd2-b1ea-7d02f1115b82 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:30.204886 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c99f4136-1001-4fd2-b1ea-7d02f1115b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:30.349806 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1152/2320] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:30.360266 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144b576-09af-47cf-98bd-a459e00d353a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:30.361797 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a144b576-09af-47cf-98bd-a459e00d353a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:30.416796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:30.449467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144b576-09af-47cf-98bd-a459e00d353a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:30.454424 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c5471-29d3-4e17-98e0-be1226bda5ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:30.455396 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c5471-29d3-4e17-98e0-be1226bda5ea service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:30.455826 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c5471-29d3-4e17-98e0-be1226bda5ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:30.456265 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c5471-29d3-4e17-98e0-be1226bda5ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:30.467077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c5471-29d3-4e17-98e0-be1226bda5ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:30.468747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c5471-29d3-4e17-98e0-be1226bda5ea service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:30.482444 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1169/2321] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:30.799242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f4136-1001-4fd2-b1ea-7d02f1115b82 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:30.800802 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c99f4136-1001-4fd2-b1ea-7d02f1115b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:30.817393 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f4136-1001-4fd2-b1ea-7d02f1115b82 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:30.822308 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eab754-af9c-4787-9fc7-7af846ec8f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:30.823300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eab754-af9c-4787-9fc7-7af846ec8f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:30.823584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eab754-af9c-4787-9fc7-7af846ec8f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:30.823860 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eab754-af9c-4787-9fc7-7af846ec8f1f admin admin] PATH_INFO: `/v3/projects/e682d32da6f24367b13bad53c448f0a0` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:30.825090 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eab754-af9c-4787-9fc7-7af846ec8f1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e682d32da6f24367b13bad53c448f0a0)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:30.827039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eab754-af9c-4787-9fc7-7af846ec8f1f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:30.877988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f0eab754-af9c-4787-9fc7-7af846ec8f1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e682d32da6f24367b13bad53c448f0a0', 'request_id': 'req-f0eab754-af9c-4787-9fc7-7af846ec8f1f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:30.887651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f0eab754-af9c-4787-9fc7-7af846ec8f1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e682d32da6f24367b13bad53c448f0a0', 'request_id': 'req-f0eab754-af9c-4787-9fc7-7af846ec8f1f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:30.895734 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f0eab754-af9c-4787-9fc7-7af846ec8f1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e682d32da6f24367b13bad53c448f0a0', 'request_id': 'req-f0eab754-af9c-4787-9fc7-7af846ec8f1f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:30.910076 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f0eab754-af9c-4787-9fc7-7af846ec8f1f admin admin] The token cache is being invalidate because project e682d32da6f24367b13bad53c448f0a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:02:30.910859 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f0eab754-af9c-4787-9fc7-7af846ec8f1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:30.911446 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f0eab754-af9c-4787-9fc7-7af846ec8f1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:30.913305 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1153/2322] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:02:30 2022] DELETE /identity/v3/projects/e682d32da6f24367b13bad53c448f0a0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:30.918597 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13967758-a37b-4440-a362-fea9fbc7cfe0 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:30.919055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13967758-a37b-4440-a362-fea9fbc7cfe0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:30.919432 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13967758-a37b-4440-a362-fea9fbc7cfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:30.949754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-13967758-a37b-4440-a362-fea9fbc7cfe0 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:30.951140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-13967758-a37b-4440-a362-fea9fbc7cfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:31.020651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1170/2323] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:31.030737 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40add6e3-5041-4bbe-b4a0-07b79f01d3d0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:31.031309 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40add6e3-5041-4bbe-b4a0-07b79f01d3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:31.031804 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40add6e3-5041-4bbe-b4a0-07b79f01d3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:31.053473 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-40add6e3-5041-4bbe-b4a0-07b79f01d3d0 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:31.054725 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-40add6e3-5041-4bbe-b4a0-07b79f01d3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:31.071708 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1154/2324] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:02:31.072496 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13967758-a37b-4440-a362-fea9fbc7cfe0 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:31.091203 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13967758-a37b-4440-a362-fea9fbc7cfe0 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:31.105353 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85733e6f-52af-42a3-b385-3c292be0b83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:31.106371 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85733e6f-52af-42a3-b385-3c292be0b83f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:31.106682 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85733e6f-52af-42a3-b385-3c292be0b83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:31.106962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85733e6f-52af-42a3-b385-3c292be0b83f admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:31.107695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85733e6f-52af-42a3-b385-3c292be0b83f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:02:31.110163 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85733e6f-52af-42a3-b385-3c292be0b83f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:31.111985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85733e6f-52af-42a3-b385-3c292be0b83f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:31.130630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1171/2325] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:02:31.132215 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40add6e3-5041-4bbe-b4a0-07b79f01d3d0 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:31.147212 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40add6e3-5041-4bbe-b4a0-07b79f01d3d0 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:31.160844 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07e5d4-8d25-41ba-b72a-6b9a5930f9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:31.161555 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07e5d4-8d25-41ba-b72a-6b9a5930f9c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:31.161841 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07e5d4-8d25-41ba-b72a-6b9a5930f9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:31.162117 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07e5d4-8d25-41ba-b72a-6b9a5930f9c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:31.162863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07e5d4-8d25-41ba-b72a-6b9a5930f9c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:31.164788 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07e5d4-8d25-41ba-b72a-6b9a5930f9c4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:31.198462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1155/2326] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:02:31 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:31.199694 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85733e6f-52af-42a3-b385-3c292be0b83f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:31.210541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85733e6f-52af-42a3-b385-3c292be0b83f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:31.274839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20b1aa-ce90-4a2a-8c82-1833f0d32d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:31.276656 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20b1aa-ce90-4a2a-8c82-1833f0d32d32 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:31.277074 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20b1aa-ce90-4a2a-8c82-1833f0d32d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:31.277396 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20b1aa-ce90-4a2a-8c82-1833f0d32d32 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:31.278469 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20b1aa-ce90-4a2a-8c82-1833f0d32d32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:31.282154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20b1aa-ce90-4a2a-8c82-1833f0d32d32 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:31.308041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1172/2327] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:02:31 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:31.312203 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07e5d4-8d25-41ba-b72a-6b9a5930f9c4 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:31.324391 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07e5d4-8d25-41ba-b72a-6b9a5930f9c4 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:31.329166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7a32d-93e2-4e50-885f-bb7f57e6ca54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:31.329824 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7a32d-93e2-4e50-885f-bb7f57e6ca54 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:31.330105 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7a32d-93e2-4e50-885f-bb7f57e6ca54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:31.330377 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7a32d-93e2-4e50-885f-bb7f57e6ca54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:31.331130 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b7a32d-93e2-4e50-885f-bb7f57e6ca54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:31.332730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b7a32d-93e2-4e50-885f-bb7f57e6ca54 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:31.344581 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1156/2328] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:31.348546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20b1aa-ce90-4a2a-8c82-1833f0d32d32 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:31.359538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20b1aa-ce90-4a2a-8c82-1833f0d32d32 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:31.364361 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94f7cd-04da-437f-9aa5-ceb34e70e989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:31.365016 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94f7cd-04da-437f-9aa5-ceb34e70e989 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:31.365299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94f7cd-04da-437f-9aa5-ceb34e70e989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:31.365574 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94f7cd-04da-437f-9aa5-ceb34e70e989 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:31.366288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94f7cd-04da-437f-9aa5-ceb34e70e989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:31.367907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94f7cd-04da-437f-9aa5-ceb34e70e989 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:31.378557 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1173/2329] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:31.382744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7a32d-93e2-4e50-885f-bb7f57e6ca54 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:31.394672 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7a32d-93e2-4e50-885f-bb7f57e6ca54 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:31.399357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515a5fe-1a1c-4a2b-9bf9-fff57d7d3f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:31.401279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515a5fe-1a1c-4a2b-9bf9-fff57d7d3f50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:31.401648 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515a5fe-1a1c-4a2b-9bf9-fff57d7d3f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:31.401923 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515a5fe-1a1c-4a2b-9bf9-fff57d7d3f50 admin admin] PATH_INFO: `/v3/projects/27d9b34e904742fa813bca967a0c8802/users/770045db01ed47e7912441fcb234117b/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:31.417630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515a5fe-1a1c-4a2b-9bf9-fff57d7d3f50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d9b34e904742fa813bca967a0c8802, user_id=770045db01ed47e7912441fcb234117b, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:31.420327 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515a5fe-1a1c-4a2b-9bf9-fff57d7d3f50 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:31.433649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1157/2330] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:02:31 2022] PUT /identity/v3/projects/27d9b34e904742fa813bca967a0c8802/users/770045db01ed47e7912441fcb234117b/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:31.443997 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7334e-1b94-4418-8413-e0f6131a27a6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:31.444296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7334e-1b94-4418-8413-e0f6131a27a6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:31.444577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7334e-1b94-4418-8413-e0f6131a27a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:31.473949 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c6a7334e-1b94-4418-8413-e0f6131a27a6 None None] MFA Rules not processed for user `770045db01ed47e7912441fcb234117b`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:31.475193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c6a7334e-1b94-4418-8413-e0f6131a27a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:31.512761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:31.550021 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1174/2331] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:31.554468 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515a5fe-1a1c-4a2b-9bf9-fff57d7d3f50 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:31.565097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:31.673426 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515a5fe-1a1c-4a2b-9bf9-fff57d7d3f50 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:31.686482 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83eb63-c9b3-486b-9cad-4816b1dc31a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:31.687219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83eb63-c9b3-486b-9cad-4816b1dc31a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:31.687506 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83eb63-c9b3-486b-9cad-4816b1dc31a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:31.687783 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83eb63-c9b3-486b-9cad-4816b1dc31a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:31.688491 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83eb63-c9b3-486b-9cad-4816b1dc31a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:31.690643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83eb63-c9b3-486b-9cad-4816b1dc31a1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:31.730752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1158/2332] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:02:31 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:31.732342 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7334e-1b94-4418-8413-e0f6131a27a6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:31.748985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7334e-1b94-4418-8413-e0f6131a27a6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:31.805151 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b273fd-5b05-417d-a8bc-2733ecef349d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:31.805526 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b273fd-5b05-417d-a8bc-2733ecef349d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:31.805877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b273fd-5b05-417d-a8bc-2733ecef349d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:31.839527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79be79-9ccb-470b-b38e-93ea513baa9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:31.840004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79be79-9ccb-470b-b38e-93ea513baa9b admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:31.840290 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79be79-9ccb-470b-b38e-93ea513baa9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:31.840569 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79be79-9ccb-470b-b38e-93ea513baa9b admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:31.841274 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79be79-9ccb-470b-b38e-93ea513baa9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:31.843263 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79be79-9ccb-470b-b38e-93ea513baa9b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:31.864989 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-82b273fd-5b05-417d-a8bc-2733ecef349d None None] MFA Rules not processed for user `02c826fc0baa480393df4c884c9e35dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:31.866196 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-82b273fd-5b05-417d-a8bc-2733ecef349d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:31.871345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1175/2333] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:02:31 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:02:31.873396 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79be79-9ccb-470b-b38e-93ea513baa9b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:31.883851 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79be79-9ccb-470b-b38e-93ea513baa9b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:31.889238 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317dc20-1212-4613-aa53-0be7f5b94924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:31.889706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317dc20-1212-4613-aa53-0be7f5b94924 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:31.890666 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317dc20-1212-4613-aa53-0be7f5b94924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:31.890666 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317dc20-1212-4613-aa53-0be7f5b94924 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:31.891824 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317dc20-1212-4613-aa53-0be7f5b94924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:31.893298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317dc20-1212-4613-aa53-0be7f5b94924 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:31.907150 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1176/2334] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:02:31.910898 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317dc20-1212-4613-aa53-0be7f5b94924 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:31.925334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317dc20-1212-4613-aa53-0be7f5b94924 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:31.930139 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d03af0-ba2e-4b90-abbd-8213d0cf790a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:31.930988 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d03af0-ba2e-4b90-abbd-8213d0cf790a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:31.931910 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d03af0-ba2e-4b90-abbd-8213d0cf790a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:31.932188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d03af0-ba2e-4b90-abbd-8213d0cf790a admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:31.932925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d03af0-ba2e-4b90-abbd-8213d0cf790a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:31.935715 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d03af0-ba2e-4b90-abbd-8213d0cf790a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:31.946045 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1159/2335] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Jun 16 19:02:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:31.948223 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1177/2336] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:31.951965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b273fd-5b05-417d-a8bc-2733ecef349d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:31.965513 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d03af0-ba2e-4b90-abbd-8213d0cf790a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:31.965996 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b273fd-5b05-417d-a8bc-2733ecef349d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:31.967678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-31d03af0-ba2e-4b90-abbd-8213d0cf790a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:31.970954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bf1d1-2306-4e0b-a190-00e28417e4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:31.972029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bf1d1-2306-4e0b-a190-00e28417e4f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:31.972386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bf1d1-2306-4e0b-a190-00e28417e4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:31.972723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bf1d1-2306-4e0b-a190-00e28417e4f5 admin admin] PATH_INFO: `/v3/projects/231f30dd95b54af5ba0f7d1516575ca5/users/59d84a18d56b443a8eea9f2ec9e58193/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:31.990255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013bf1d1-2306-4e0b-a190-00e28417e4f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=231f30dd95b54af5ba0f7d1516575ca5, user_id=59d84a18d56b443a8eea9f2ec9e58193, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:31.993148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013bf1d1-2306-4e0b-a190-00e28417e4f5 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:32.018651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1160/2337] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:02:31 2022] PUT /identity/v3/projects/231f30dd95b54af5ba0f7d1516575ca5/users/59d84a18d56b443a8eea9f2ec9e58193/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:32.024353 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5756ec-d070-4b85-808e-4dc288f738ec None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:32.024652 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5756ec-d070-4b85-808e-4dc288f738ec None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:32.026167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5756ec-d070-4b85-808e-4dc288f738ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:32.059363 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-9c5756ec-d070-4b85-808e-4dc288f738ec None None] MFA Rules not processed for user `59d84a18d56b443a8eea9f2ec9e58193`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:32.059903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9c5756ec-d070-4b85-808e-4dc288f738ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:32.092441 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d03af0-ba2e-4b90-abbd-8213d0cf790a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:32.097252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:32.117463 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:32.152489 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1161/2338] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:02:32.153458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5756ec-d070-4b85-808e-4dc288f738ec None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:32.194453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac528a2-33dc-4dc8-a196-4c313b7f6ea1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b22fe0ab0354385924923d0d443acae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:32.196128 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac528a2-33dc-4dc8-a196-4c313b7f6ea1 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:32.196128 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac528a2-33dc-4dc8-a196-4c313b7f6ea1 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:32.196128 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac528a2-33dc-4dc8-a196-4c313b7f6ea1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:32.234105 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5756ec-d070-4b85-808e-4dc288f738ec None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:32.240975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5563ab-ec3b-4c1f-b1b8-6d46078fcbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:32.241640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5563ab-ec3b-4c1f-b1b8-6d46078fcbe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:32.241922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5563ab-ec3b-4c1f-b1b8-6d46078fcbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:32.242197 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5563ab-ec3b-4c1f-b1b8-6d46078fcbe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:32.242915 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5563ab-ec3b-4c1f-b1b8-6d46078fcbe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:32.246279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5563ab-ec3b-4c1f-b1b8-6d46078fcbe2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:32.266748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac528a2-33dc-4dc8-a196-4c313b7f6ea1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:32.268962 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac528a2-33dc-4dc8-a196-4c313b7f6ea1 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:32.275836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1162/2339] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:02:32 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:32.279470 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1178/2340] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Jun 16 19:02:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 19:02:32.280024 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5563ab-ec3b-4c1f-b1b8-6d46078fcbe2 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:32.361598 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5563ab-ec3b-4c1f-b1b8-6d46078fcbe2 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:32.367296 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d27bc-b1da-43b1-a891-b4c1dc7e6921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:32.368253 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d27bc-b1da-43b1-a891-b4c1dc7e6921 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:32.368661 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d27bc-b1da-43b1-a891-b4c1dc7e6921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:32.368993 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d27bc-b1da-43b1-a891-b4c1dc7e6921 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:32.369893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623d27bc-b1da-43b1-a891-b4c1dc7e6921 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:32.371923 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623d27bc-b1da-43b1-a891-b4c1dc7e6921 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:32.398861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1163/2341] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:02:32 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:32.401766 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac528a2-33dc-4dc8-a196-4c313b7f6ea1 service swift] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:32.413059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac528a2-33dc-4dc8-a196-4c313b7f6ea1 service swift] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:32.418309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f7eb0-f26c-4f46-b978-5b35c15fbbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:32.419018 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f7eb0-f26c-4f46-b978-5b35c15fbbd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:32.419299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f7eb0-f26c-4f46-b978-5b35c15fbbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:32.419571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f7eb0-f26c-4f46-b978-5b35c15fbbd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:32.420532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f7eb0-f26c-4f46-b978-5b35c15fbbd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:32.422130 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f7eb0-f26c-4f46-b978-5b35c15fbbd0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:32.436065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1179/2342] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:02:32.437312 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d27bc-b1da-43b1-a891-b4c1dc7e6921 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:32.449264 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d27bc-b1da-43b1-a891-b4c1dc7e6921 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:32.454127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4146f-432a-45b7-847d-0ea94621e16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:32.455052 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4146f-432a-45b7-847d-0ea94621e16c admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:32.455339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4146f-432a-45b7-847d-0ea94621e16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:32.455615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4146f-432a-45b7-847d-0ea94621e16c admin admin] PATH_INFO: `/v3/projects/59b447c810444ef299c26863b4c70ad6/users/5d9046eba61348a585a13e651ea73664/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:32.472293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4146f-432a-45b7-847d-0ea94621e16c admin admin] RBAC: Authorizing `identity:create_grant(project_id=59b447c810444ef299c26863b4c70ad6, user_id=5d9046eba61348a585a13e651ea73664, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:32.475359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4146f-432a-45b7-847d-0ea94621e16c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:32.490099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1164/2343] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:02:32 2022] PUT /identity/v3/projects/59b447c810444ef299c26863b4c70ad6/users/5d9046eba61348a585a13e651ea73664/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:32.494398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f7eb0-f26c-4f46-b978-5b35c15fbbd0 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:32.504631 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f7eb0-f26c-4f46-b978-5b35c15fbbd0 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:32.582527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc8ffb-0cff-4424-a9b7-e8d780c06021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:32.583316 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc8ffb-0cff-4424-a9b7-e8d780c06021 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:32.583600 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc8ffb-0cff-4424-a9b7-e8d780c06021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:32.583876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc8ffb-0cff-4424-a9b7-e8d780c06021 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:32.584605 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc8ffb-0cff-4424-a9b7-e8d780c06021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:32.586280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc8ffb-0cff-4424-a9b7-e8d780c06021 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:32.601102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1180/2344] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:32.602379 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4146f-432a-45b7-847d-0ea94621e16c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:32.613594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4146f-432a-45b7-847d-0ea94621e16c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:32.619843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14674b70-a74c-4253-940b-484c26df900c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:32.621277 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14674b70-a74c-4253-940b-484c26df900c admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:32.621548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14674b70-a74c-4253-940b-484c26df900c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:32.621752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14674b70-a74c-4253-940b-484c26df900c admin admin] PATH_INFO: `/v3/projects/59b447c810444ef299c26863b4c70ad6/users/5d9046eba61348a585a13e651ea73664/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:32.623853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14674b70-a74c-4253-940b-484c26df900c admin admin] RBAC: Authorizing `identity:create_grant(project_id=59b447c810444ef299c26863b4c70ad6, user_id=5d9046eba61348a585a13e651ea73664, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:32.626610 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14674b70-a74c-4253-940b-484c26df900c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:32.641239 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1165/2345] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:02:32 2022] PUT /identity/v3/projects/59b447c810444ef299c26863b4c70ad6/users/5d9046eba61348a585a13e651ea73664/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:32.653618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2735a-93f6-482a-8bc6-fa9db2afb570 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:32.654617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2735a-93f6-482a-8bc6-fa9db2afb570 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:32.654617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2735a-93f6-482a-8bc6-fa9db2afb570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:32.694837 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-05d2735a-93f6-482a-8bc6-fa9db2afb570 None None] MFA Rules not processed for user `5d9046eba61348a585a13e651ea73664`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:32.696123 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-05d2735a-93f6-482a-8bc6-fa9db2afb570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:32.744164 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:32.783288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1181/2346] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:32.803402 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14674b70-a74c-4253-940b-484c26df900c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:32.808009 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-14674b70-a74c-4253-940b-484c26df900c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:32.821424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:32.928383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14674b70-a74c-4253-940b-484c26df900c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:32.938389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d370ca-ac95-41fc-9e77-74e6c6eac562 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:32.939315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d370ca-ac95-41fc-9e77-74e6c6eac562 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:32.939768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d370ca-ac95-41fc-9e77-74e6c6eac562 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:32.940272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d370ca-ac95-41fc-9e77-74e6c6eac562 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:32.943815 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0d5bf-9371-45b2-8143-40cda252bd40 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:32.944093 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0d5bf-9371-45b2-8143-40cda252bd40 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:32.944372 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0d5bf-9371-45b2-8143-40cda252bd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:32.968464 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6ae0d5bf-9371-45b2-8143-40cda252bd40 None None] MFA Rules not processed for user `02c826fc0baa480393df4c884c9e35dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:32.969832 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6ae0d5bf-9371-45b2-8143-40cda252bd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:33.035462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1182/2347] 10.222.0.70 () {58 vars in 967 bytes} [Thu Jun 16 19:02:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:33.039124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d370ca-ac95-41fc-9e77-74e6c6eac562 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:33.041467 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d370ca-ac95-41fc-9e77-74e6c6eac562 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:33.052524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1166/2348] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:33.063076 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0d5bf-9371-45b2-8143-40cda252bd40 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:33.126420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a66e01-85d3-44f2-ae71-0559d6ee395f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:33.126777 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a66e01-85d3-44f2-ae71-0559d6ee395f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:33.127287 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a66e01-85d3-44f2-ae71-0559d6ee395f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:33.151384 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c2a66e01-85d3-44f2-ae71-0559d6ee395f None None] MFA Rules not processed for user `02c826fc0baa480393df4c884c9e35dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:33.153224 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c2a66e01-85d3-44f2-ae71-0559d6ee395f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:33.164466 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1167/2349] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Jun 16 19:02:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:33.168107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0d5bf-9371-45b2-8143-40cda252bd40 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:33.179249 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e0cb8-3f23-44f3-836a-f80fe65b5c18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b22fe0ab0354385924923d0d443acae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:33.180004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e0cb8-3f23-44f3-836a-f80fe65b5c18 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:33.180294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e0cb8-3f23-44f3-836a-f80fe65b5c18 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:33.180577 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e0cb8-3f23-44f3-836a-f80fe65b5c18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:33.182884 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a66e01-85d3-44f2-ae71-0559d6ee395f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:33.193857 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e0cb8-3f23-44f3-836a-f80fe65b5c18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:33.195501 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e0cb8-3f23-44f3-836a-f80fe65b5c18 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:33.201756 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a66e01-85d3-44f2-ae71-0559d6ee395f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:33.209541 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f0feb-8f4a-49fa-a931-5725fe43243f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b22fe0ab0354385924923d0d443acae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:33.210460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f0feb-8f4a-49fa-a931-5725fe43243f service swift] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:33.210838 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f0feb-8f4a-49fa-a931-5725fe43243f service swift] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:33.211190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f0feb-8f4a-49fa-a931-5725fe43243f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:33.214029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1183/2350] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Jun 16 19:02:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:02:33.221188 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f0feb-8f4a-49fa-a931-5725fe43243f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:33.222870 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f0feb-8f4a-49fa-a931-5725fe43243f service swift] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:33.237943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1168/2351] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Jun 16 19:02:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 19:02:33.594149 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489d8cc-8b75-4bc3-8a18-105195219897 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:33.594745 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489d8cc-8b75-4bc3-8a18-105195219897 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:33.595196 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489d8cc-8b75-4bc3-8a18-105195219897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:33.625854 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f489d8cc-8b75-4bc3-8a18-105195219897 None None] MFA Rules not processed for user `02c826fc0baa480393df4c884c9e35dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:33.627282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f489d8cc-8b75-4bc3-8a18-105195219897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:33.636910 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1184/2352] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Jun 16 19:02:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:33.665528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f0feb-8f4a-49fa-a931-5725fe43243f service swift] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:33.678245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f0feb-8f4a-49fa-a931-5725fe43243f service swift] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:33.683982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1ce1a-065e-4c6a-b8c3-09999ff600e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b22fe0ab0354385924923d0d443acae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:33.684701 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1ce1a-065e-4c6a-b8c3-09999ff600e9 service swift] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:33.685186 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1ce1a-065e-4c6a-b8c3-09999ff600e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:33.685643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1ce1a-065e-4c6a-b8c3-09999ff600e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:33.696503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1ce1a-065e-4c6a-b8c3-09999ff600e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:33.699861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1ce1a-065e-4c6a-b8c3-09999ff600e9 service swift] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:33.710832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1169/2353] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Jun 16 19:02:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:02:34.167221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea33b8-fe3c-4f31-9741-76e433cb738e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:34.168142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea33b8-fe3c-4f31-9741-76e433cb738e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:34.168626 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea33b8-fe3c-4f31-9741-76e433cb738e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:34.189303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-31ea33b8-fe3c-4f31-9741-76e433cb738e None None] MFA Rules not processed for user `02c826fc0baa480393df4c884c9e35dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:34.190853 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-31ea33b8-fe3c-4f31-9741-76e433cb738e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:34.198429 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1185/2354] 10.222.0.70 () {58 vars in 967 bytes} [Thu Jun 16 19:02:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:34.222050 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1ce1a-065e-4c6a-b8c3-09999ff600e9 service swift] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:34.234200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1ce1a-065e-4c6a-b8c3-09999ff600e9 service swift] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:34.239413 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ba0b7-a8b5-491f-a268-baeac8f23b43 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b22fe0ab0354385924923d0d443acae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:34.240087 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ba0b7-a8b5-491f-a268-baeac8f23b43 service swift] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:34.240456 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ba0b7-a8b5-491f-a268-baeac8f23b43 service swift] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:34.240736 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ba0b7-a8b5-491f-a268-baeac8f23b43 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:34.249661 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ba0b7-a8b5-491f-a268-baeac8f23b43 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:34.251301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ba0b7-a8b5-491f-a268-baeac8f23b43 service swift] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:34.259777 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1170/2355] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Jun 16 19:02:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:02:34.786563 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea33b8-fe3c-4f31-9741-76e433cb738e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:34.804472 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea33b8-fe3c-4f31-9741-76e433cb738e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:34.807191 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d98fd-38d3-4be0-b9da-e67325a11471 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:34.808084 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d98fd-38d3-4be0-b9da-e67325a11471 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:34.808372 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d98fd-38d3-4be0-b9da-e67325a11471 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:34.808652 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d98fd-38d3-4be0-b9da-e67325a11471 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:34.895639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:34.903735 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d98fd-38d3-4be0-b9da-e67325a11471 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:34.905314 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d98fd-38d3-4be0-b9da-e67325a11471 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:34.920514 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1186/2356] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:35.710977 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ba0b7-a8b5-491f-a268-baeac8f23b43 service swift] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:35.712600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-172ba0b7-a8b5-491f-a268-baeac8f23b43 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:35.725040 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ba0b7-a8b5-491f-a268-baeac8f23b43 service swift] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:35.728821 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59797a13-9009-4469-9f8a-7699db1d3f76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:35.729464 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59797a13-9009-4469-9f8a-7699db1d3f76 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:35.729775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59797a13-9009-4469-9f8a-7699db1d3f76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:35.730056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59797a13-9009-4469-9f8a-7699db1d3f76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:35.740266 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59797a13-9009-4469-9f8a-7699db1d3f76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:35.741695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59797a13-9009-4469-9f8a-7699db1d3f76 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:35.752212 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1171/2357] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:36.632475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d98fd-38d3-4be0-b9da-e67325a11471 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:36.633985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ef3d98fd-38d3-4be0-b9da-e67325a11471 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:36.725096 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d98fd-38d3-4be0-b9da-e67325a11471 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:36.731292 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce792c8-9b66-49e8-87d1-1d2b00436114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:36.732528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce792c8-9b66-49e8-87d1-1d2b00436114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:36.733013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce792c8-9b66-49e8-87d1-1d2b00436114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:36.733709 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce792c8-9b66-49e8-87d1-1d2b00436114 admin admin] PATH_INFO: `/v3/users/7e66f41aa3194667bd5970337fd41cfa` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:36.735543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce792c8-9b66-49e8-87d1-1d2b00436114 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e66f41aa3194667bd5970337fd41cfa)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:36.738616 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce792c8-9b66-49e8-87d1-1d2b00436114 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:36.803119 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6ce792c8-9b66-49e8-87d1-1d2b00436114 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e66f41aa3194667bd5970337fd41cfa'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:36.804072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6ce792c8-9b66-49e8-87d1-1d2b00436114 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e66f41aa3194667bd5970337fd41cfa'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:36.804839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6ce792c8-9b66-49e8-87d1-1d2b00436114 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e66f41aa3194667bd5970337fd41cfa'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:36.814710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6ce792c8-9b66-49e8-87d1-1d2b00436114 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e66f41aa3194667bd5970337fd41cfa'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:36.825418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6ce792c8-9b66-49e8-87d1-1d2b00436114 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e66f41aa3194667bd5970337fd41cfa'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:36.851100 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1187/2358] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:02:36 2022] DELETE /identity/v3/users/7e66f41aa3194667bd5970337fd41cfa => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:36.856768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a12ab-b9d0-44b5-a9a3-89140c3d9618 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:36.857270 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a12ab-b9d0-44b5-a9a3-89140c3d9618 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:36.857360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a12ab-b9d0-44b5-a9a3-89140c3d9618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:36.878996 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-659a12ab-b9d0-44b5-a9a3-89140c3d9618 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:36.880246 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-659a12ab-b9d0-44b5-a9a3-89140c3d9618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:36.968344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1172/2359] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:36.978641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce792c8-9b66-49e8-87d1-1d2b00436114 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:36.980318 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6ce792c8-9b66-49e8-87d1-1d2b00436114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:37.051335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce792c8-9b66-49e8-87d1-1d2b00436114 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:37.056863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d1e8b-8589-4801-b48a-f4c0d9f471e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:37.057541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d1e8b-8589-4801-b48a-f4c0d9f471e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.057897 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d1e8b-8589-4801-b48a-f4c0d9f471e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.058198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d1e8b-8589-4801-b48a-f4c0d9f471e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.069030 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d1e8b-8589-4801-b48a-f4c0d9f471e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:37.070630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d1e8b-8589-4801-b48a-f4c0d9f471e1 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:37.082141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1188/2360] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:37.392472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a12ab-b9d0-44b5-a9a3-89140c3d9618 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:37.393921 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-659a12ab-b9d0-44b5-a9a3-89140c3d9618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:37.403189 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:37.419975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a12ab-b9d0-44b5-a9a3-89140c3d9618 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:37.428240 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058f3b1-f88c-453a-8ab2-9a2b02383844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:37.429213 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058f3b1-f88c-453a-8ab2-9a2b02383844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.429383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058f3b1-f88c-453a-8ab2-9a2b02383844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.429983 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058f3b1-f88c-453a-8ab2-9a2b02383844 admin admin] PATH_INFO: `/v3/projects/bae3331bfc29456c94e309c51cb82a9f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.431500 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058f3b1-f88c-453a-8ab2-9a2b02383844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bae3331bfc29456c94e309c51cb82a9f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:37.433664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058f3b1-f88c-453a-8ab2-9a2b02383844 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:37.488633 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d058f3b1-f88c-453a-8ab2-9a2b02383844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bae3331bfc29456c94e309c51cb82a9f', 'request_id': 'req-d058f3b1-f88c-453a-8ab2-9a2b02383844'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:37.497247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d058f3b1-f88c-453a-8ab2-9a2b02383844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bae3331bfc29456c94e309c51cb82a9f', 'request_id': 'req-d058f3b1-f88c-453a-8ab2-9a2b02383844'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:37.503944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d058f3b1-f88c-453a-8ab2-9a2b02383844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bae3331bfc29456c94e309c51cb82a9f', 'request_id': 'req-d058f3b1-f88c-453a-8ab2-9a2b02383844'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:37.522611 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d058f3b1-f88c-453a-8ab2-9a2b02383844 admin admin] The token cache is being invalidate because project bae3331bfc29456c94e309c51cb82a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:02:37.522922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d058f3b1-f88c-453a-8ab2-9a2b02383844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:37.523624 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d058f3b1-f88c-453a-8ab2-9a2b02383844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:37.526469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1173/2361] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:02:37 2022] DELETE /identity/v3/projects/bae3331bfc29456c94e309c51cb82a9f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:37.531493 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6fae4-c576-4d5e-aff0-271350c786c5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.531793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6fae4-c576-4d5e-aff0-271350c786c5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.532071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6fae4-c576-4d5e-aff0-271350c786c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.551399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d4c6fae4-c576-4d5e-aff0-271350c786c5 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:37.552603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d4c6fae4-c576-4d5e-aff0-271350c786c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:37.567436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:37.623130 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1189/2362] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:37.630634 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013353b-a435-48d2-8fc1-634a6ae3b8d0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.630937 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013353b-a435-48d2-8fc1-634a6ae3b8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.631221 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013353b-a435-48d2-8fc1-634a6ae3b8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.652167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c013353b-a435-48d2-8fc1-634a6ae3b8d0 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:37.653462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c013353b-a435-48d2-8fc1-634a6ae3b8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:37.661536 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1174/2363] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:37.666055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6fae4-c576-4d5e-aff0-271350c786c5 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:37.676355 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6fae4-c576-4d5e-aff0-271350c786c5 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:37.680951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf53bb-3853-40ed-8a6c-f239f61da5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:37.681610 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf53bb-3853-40ed-8a6c-f239f61da5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.681975 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf53bb-3853-40ed-8a6c-f239f61da5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.682262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf53bb-3853-40ed-8a6c-f239f61da5b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.683080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf53bb-3853-40ed-8a6c-f239f61da5b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:02:37.683388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf53bb-3853-40ed-8a6c-f239f61da5b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:37.684981 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf53bb-3853-40ed-8a6c-f239f61da5b4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:37.699578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1190/2364] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:02:37.703983 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013353b-a435-48d2-8fc1-634a6ae3b8d0 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:37.714812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013353b-a435-48d2-8fc1-634a6ae3b8d0 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:37.720845 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1b931-4e75-43c9-912b-ceff56fa99c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:37.721599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1b931-4e75-43c9-912b-ceff56fa99c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.721870 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1b931-4e75-43c9-912b-ceff56fa99c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.722165 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1b931-4e75-43c9-912b-ceff56fa99c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.722903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1b931-4e75-43c9-912b-ceff56fa99c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:37.724822 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1b931-4e75-43c9-912b-ceff56fa99c1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:37.755533 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1175/2365] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:02:37 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:37.756792 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf53bb-3853-40ed-8a6c-f239f61da5b4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:37.768757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf53bb-3853-40ed-8a6c-f239f61da5b4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:37.834700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb05a41-89b5-45e8-be37-0695daf5d96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:37.835377 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb05a41-89b5-45e8-be37-0695daf5d96b admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.835687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb05a41-89b5-45e8-be37-0695daf5d96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.835969 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb05a41-89b5-45e8-be37-0695daf5d96b admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.836659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb05a41-89b5-45e8-be37-0695daf5d96b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:37.839460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb05a41-89b5-45e8-be37-0695daf5d96b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:37.862149 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1191/2366] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:02:37 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:37.866652 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1b931-4e75-43c9-912b-ceff56fa99c1 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:37.878407 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1b931-4e75-43c9-912b-ceff56fa99c1 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:37.883106 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20109872-20c7-4545-b58e-24cde4dd5cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:37.883766 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20109872-20c7-4545-b58e-24cde4dd5cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.884120 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20109872-20c7-4545-b58e-24cde4dd5cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.884489 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20109872-20c7-4545-b58e-24cde4dd5cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.885205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20109872-20c7-4545-b58e-24cde4dd5cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:37.886891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20109872-20c7-4545-b58e-24cde4dd5cf6 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:37.897972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1176/2367] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:37.902889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb05a41-89b5-45e8-be37-0695daf5d96b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:37.912800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb05a41-89b5-45e8-be37-0695daf5d96b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:37.918618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468e389-87c9-4de1-92ce-23da96b33b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:37.919313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468e389-87c9-4de1-92ce-23da96b33b7f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.919567 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468e389-87c9-4de1-92ce-23da96b33b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.919845 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468e389-87c9-4de1-92ce-23da96b33b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.920565 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468e389-87c9-4de1-92ce-23da96b33b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:37.922204 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468e389-87c9-4de1-92ce-23da96b33b7f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:37.934261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1192/2368] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:37.939121 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20109872-20c7-4545-b58e-24cde4dd5cf6 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:37.949977 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20109872-20c7-4545-b58e-24cde4dd5cf6 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:37.957527 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6ce87-70db-4e08-9241-c077e570eb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:37.958800 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6ce87-70db-4e08-9241-c077e570eb9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.959445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6ce87-70db-4e08-9241-c077e570eb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.959944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6ce87-70db-4e08-9241-c077e570eb9e admin admin] PATH_INFO: `/v3/projects/ddb2c879d7f44189ab76e377c33f5e35/users/fb08a57a393d4c8591da42cd5e92aca9/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.986360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6ce87-70db-4e08-9241-c077e570eb9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddb2c879d7f44189ab76e377c33f5e35, user_id=fb08a57a393d4c8591da42cd5e92aca9, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:37.989625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6ce87-70db-4e08-9241-c077e570eb9e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:38.004893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1177/2369] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:02:37 2022] PUT /identity/v3/projects/ddb2c879d7f44189ab76e377c33f5e35/users/fb08a57a393d4c8591da42cd5e92aca9/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:38.010300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1c730-c2de-4132-b1f8-3687ccc554ac None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:38.010885 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1c730-c2de-4132-b1f8-3687ccc554ac None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:38.011287 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1c730-c2de-4132-b1f8-3687ccc554ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:38.032886 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e9b1c730-c2de-4132-b1f8-3687ccc554ac None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:38.034615 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e9b1c730-c2de-4132-b1f8-3687ccc554ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:38.113409 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1193/2370] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:38.123013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6ce87-70db-4e08-9241-c077e570eb9e admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:38.124777 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-24d6ce87-70db-4e08-9241-c077e570eb9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:38.189219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6ce87-70db-4e08-9241-c077e570eb9e admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:38.193130 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0639e6b-8882-4e18-939e-46f02e0d0c6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:38.193854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0639e6b-8882-4e18-939e-46f02e0d0c6f service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:38.194137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0639e6b-8882-4e18-939e-46f02e0d0c6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:38.194411 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0639e6b-8882-4e18-939e-46f02e0d0c6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:38.204851 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0639e6b-8882-4e18-939e-46f02e0d0c6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:38.206269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0639e6b-8882-4e18-939e-46f02e0d0c6f service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:38.216355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1178/2371] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:02:39.110838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9beb4d-3e7b-42ed-936a-3901c235fd5b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:39.111490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9beb4d-3e7b-42ed-936a-3901c235fd5b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:39.111587 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9beb4d-3e7b-42ed-936a-3901c235fd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:39.177895 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3d9beb4d-3e7b-42ed-936a-3901c235fd5b None None] MFA Rules not processed for user `fb08a57a393d4c8591da42cd5e92aca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:39.179863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3d9beb4d-3e7b-42ed-936a-3901c235fd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:39.218926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:39.299394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1194/2372] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:39.311487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0639e6b-8882-4e18-939e-46f02e0d0c6f service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:39.314866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b0639e6b-8882-4e18-939e-46f02e0d0c6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:39.357219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:39.392627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0639e6b-8882-4e18-939e-46f02e0d0c6f service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:39.396468 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2694a36-3dc4-4146-8820-a6a0a75f0a7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:39.397111 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2694a36-3dc4-4146-8820-a6a0a75f0a7c service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:39.397390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2694a36-3dc4-4146-8820-a6a0a75f0a7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:39.397664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2694a36-3dc4-4146-8820-a6a0a75f0a7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:39.407803 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2694a36-3dc4-4146-8820-a6a0a75f0a7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:39.409490 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2694a36-3dc4-4146-8820-a6a0a75f0a7c service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:39.420036 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1179/2373] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:39.849907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9beb4d-3e7b-42ed-936a-3901c235fd5b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:39.851467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3d9beb4d-3e7b-42ed-936a-3901c235fd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:39.886647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9beb4d-3e7b-42ed-936a-3901c235fd5b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:39.890837 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc156027-a0b9-44a6-9353-9333b08aea31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:39.891764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc156027-a0b9-44a6-9353-9333b08aea31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:39.892057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc156027-a0b9-44a6-9353-9333b08aea31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:39.892352 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc156027-a0b9-44a6-9353-9333b08aea31 admin admin] PATH_INFO: `/v3/users/770045db01ed47e7912441fcb234117b` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:39.893591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc156027-a0b9-44a6-9353-9333b08aea31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=770045db01ed47e7912441fcb234117b)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:39.895641 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc156027-a0b9-44a6-9353-9333b08aea31 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:39.962721 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dc156027-a0b9-44a6-9353-9333b08aea31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770045db01ed47e7912441fcb234117b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:39.963453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dc156027-a0b9-44a6-9353-9333b08aea31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770045db01ed47e7912441fcb234117b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:39.964071 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dc156027-a0b9-44a6-9353-9333b08aea31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770045db01ed47e7912441fcb234117b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:39.973154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dc156027-a0b9-44a6-9353-9333b08aea31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770045db01ed47e7912441fcb234117b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:39.982493 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dc156027-a0b9-44a6-9353-9333b08aea31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770045db01ed47e7912441fcb234117b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:40.003030 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1195/2374] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:02:39 2022] DELETE /identity/v3/users/770045db01ed47e7912441fcb234117b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:40.007453 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e1417-7ebf-4dd4-8d95-3a94d7015e37 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:40.007827 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e1417-7ebf-4dd4-8d95-3a94d7015e37 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:40.008071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e1417-7ebf-4dd4-8d95-3a94d7015e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:40.027838 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-138e1417-7ebf-4dd4-8d95-3a94d7015e37 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:40.029082 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-138e1417-7ebf-4dd4-8d95-3a94d7015e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:40.098725 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1180/2375] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:40.107923 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc156027-a0b9-44a6-9353-9333b08aea31 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:40.109389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-dc156027-a0b9-44a6-9353-9333b08aea31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:40.171355 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc156027-a0b9-44a6-9353-9333b08aea31 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:40.175778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d8105-2088-4e9c-ba1e-e39f0be37a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:40.176536 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d8105-2088-4e9c-ba1e-e39f0be37a44 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:40.176827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d8105-2088-4e9c-ba1e-e39f0be37a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:40.177106 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d8105-2088-4e9c-ba1e-e39f0be37a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:40.186666 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d8105-2088-4e9c-ba1e-e39f0be37a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:40.188100 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d8105-2088-4e9c-ba1e-e39f0be37a44 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:40.198272 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1196/2376] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:40.624385 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e1417-7ebf-4dd4-8d95-3a94d7015e37 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:40.627862 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-138e1417-7ebf-4dd4-8d95-3a94d7015e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:40.646086 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e1417-7ebf-4dd4-8d95-3a94d7015e37 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:40.652818 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26589d-1aae-48a0-b993-ce87d79bf425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:40.653699 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26589d-1aae-48a0-b993-ce87d79bf425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:40.654198 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26589d-1aae-48a0-b993-ce87d79bf425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:40.654301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26589d-1aae-48a0-b993-ce87d79bf425 admin admin] PATH_INFO: `/v3/projects/27d9b34e904742fa813bca967a0c8802` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:40.655585 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26589d-1aae-48a0-b993-ce87d79bf425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27d9b34e904742fa813bca967a0c8802)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:40.657502 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26589d-1aae-48a0-b993-ce87d79bf425 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:40.711080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-fb26589d-1aae-48a0-b993-ce87d79bf425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27d9b34e904742fa813bca967a0c8802', 'request_id': 'req-fb26589d-1aae-48a0-b993-ce87d79bf425'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:40.720016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-fb26589d-1aae-48a0-b993-ce87d79bf425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27d9b34e904742fa813bca967a0c8802', 'request_id': 'req-fb26589d-1aae-48a0-b993-ce87d79bf425'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:40.727225 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-fb26589d-1aae-48a0-b993-ce87d79bf425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27d9b34e904742fa813bca967a0c8802', 'request_id': 'req-fb26589d-1aae-48a0-b993-ce87d79bf425'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:40.739641 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-fb26589d-1aae-48a0-b993-ce87d79bf425 admin admin] The token cache is being invalidate because project 27d9b34e904742fa813bca967a0c8802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:02:40.739953 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-fb26589d-1aae-48a0-b993-ce87d79bf425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:40.740665 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-fb26589d-1aae-48a0-b993-ce87d79bf425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:40.742337 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1181/2377] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:02:40 2022] DELETE /identity/v3/projects/27d9b34e904742fa813bca967a0c8802 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:40.747110 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d8105-2088-4e9c-ba1e-e39f0be37a44 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:40.761996 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d8105-2088-4e9c-ba1e-e39f0be37a44 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:40.801761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:40.841042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d7c71-6dd9-43e9-95e4-cc802b99d2d2 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:40.841330 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d7c71-6dd9-43e9-95e4-cc802b99d2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:40.841668 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d7c71-6dd9-43e9-95e4-cc802b99d2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:40.856571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9aea1-2ef2-4cc9-9bae-9708753b5ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:40.857477 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9aea1-2ef2-4cc9-9bae-9708753b5ff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:40.857835 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9aea1-2ef2-4cc9-9bae-9708753b5ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:40.858472 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9aea1-2ef2-4cc9-9bae-9708753b5ff5 admin admin] PATH_INFO: `/v3/users/59d84a18d56b443a8eea9f2ec9e58193` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:40.859737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9aea1-2ef2-4cc9-9bae-9708753b5ff5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59d84a18d56b443a8eea9f2ec9e58193)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:40.861807 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9aea1-2ef2-4cc9-9bae-9708753b5ff5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:40.862655 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ce4d7c71-6dd9-43e9-95e4-cc802b99d2d2 None None] MFA Rules not processed for user `fb08a57a393d4c8591da42cd5e92aca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:40.864073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ce4d7c71-6dd9-43e9-95e4-cc802b99d2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:40.929344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:40.950472 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1bf9aea1-2ef2-4cc9-9bae-9708753b5ff5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59d84a18d56b443a8eea9f2ec9e58193'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:40.951257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1bf9aea1-2ef2-4cc9-9bae-9708753b5ff5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59d84a18d56b443a8eea9f2ec9e58193'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:40.955908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1bf9aea1-2ef2-4cc9-9bae-9708753b5ff5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59d84a18d56b443a8eea9f2ec9e58193'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:40.972599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1182/2378] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:40.979400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1bf9aea1-2ef2-4cc9-9bae-9708753b5ff5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59d84a18d56b443a8eea9f2ec9e58193'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:40.991425 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1bf9aea1-2ef2-4cc9-9bae-9708753b5ff5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59d84a18d56b443a8eea9f2ec9e58193'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:41.016414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1197/2379] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:02:40 2022] DELETE /identity/v3/users/59d84a18d56b443a8eea9f2ec9e58193 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:41.552451 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d7c71-6dd9-43e9-95e4-cc802b99d2d2 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:41.554297 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ce4d7c71-6dd9-43e9-95e4-cc802b99d2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:41.575990 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d7c71-6dd9-43e9-95e4-cc802b99d2d2 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:41.677474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c1acc-716f-441c-bce7-eaf1b56bba23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:41.678396 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c1acc-716f-441c-bce7-eaf1b56bba23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:41.678715 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c1acc-716f-441c-bce7-eaf1b56bba23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:41.678992 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c1acc-716f-441c-bce7-eaf1b56bba23 admin admin] PATH_INFO: `/v3/projects/231f30dd95b54af5ba0f7d1516575ca5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:41.683429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c1acc-716f-441c-bce7-eaf1b56bba23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=231f30dd95b54af5ba0f7d1516575ca5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:41.685508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c1acc-716f-441c-bce7-eaf1b56bba23 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:41.772238 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3d1c1acc-716f-441c-bce7-eaf1b56bba23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '231f30dd95b54af5ba0f7d1516575ca5', 'request_id': 'req-3d1c1acc-716f-441c-bce7-eaf1b56bba23'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:41.783746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3d1c1acc-716f-441c-bce7-eaf1b56bba23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '231f30dd95b54af5ba0f7d1516575ca5', 'request_id': 'req-3d1c1acc-716f-441c-bce7-eaf1b56bba23'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:41.798249 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3d1c1acc-716f-441c-bce7-eaf1b56bba23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '231f30dd95b54af5ba0f7d1516575ca5', 'request_id': 'req-3d1c1acc-716f-441c-bce7-eaf1b56bba23'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:41.811992 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3d1c1acc-716f-441c-bce7-eaf1b56bba23 admin admin] The token cache is being invalidate because project 231f30dd95b54af5ba0f7d1516575ca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:02:41.812458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3d1c1acc-716f-441c-bce7-eaf1b56bba23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:41.813608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3d1c1acc-716f-441c-bce7-eaf1b56bba23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:41.818064 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1183/2380] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:02:41 2022] DELETE /identity/v3/projects/231f30dd95b54af5ba0f7d1516575ca5 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:41.819282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9aea1-2ef2-4cc9-9bae-9708753b5ff5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:41.831032 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9aea1-2ef2-4cc9-9bae-9708753b5ff5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:41.920310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288aed6-063f-44ee-ae5e-48b87ecdfde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:41.921632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288aed6-063f-44ee-ae5e-48b87ecdfde6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:41.921960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288aed6-063f-44ee-ae5e-48b87ecdfde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:41.922297 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288aed6-063f-44ee-ae5e-48b87ecdfde6 admin admin] PATH_INFO: `/v3/users/5d9046eba61348a585a13e651ea73664` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:41.924023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288aed6-063f-44ee-ae5e-48b87ecdfde6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d9046eba61348a585a13e651ea73664)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:41.934751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288aed6-063f-44ee-ae5e-48b87ecdfde6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:42.033602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4288aed6-063f-44ee-ae5e-48b87ecdfde6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d9046eba61348a585a13e651ea73664'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:42.034359 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4288aed6-063f-44ee-ae5e-48b87ecdfde6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d9046eba61348a585a13e651ea73664'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:42.035592 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4288aed6-063f-44ee-ae5e-48b87ecdfde6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d9046eba61348a585a13e651ea73664'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:42.040280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:42.048808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4288aed6-063f-44ee-ae5e-48b87ecdfde6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d9046eba61348a585a13e651ea73664'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:42.058288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4288aed6-063f-44ee-ae5e-48b87ecdfde6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d9046eba61348a585a13e651ea73664'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:42.086222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1198/2381] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:02:41 2022] DELETE /identity/v3/users/5d9046eba61348a585a13e651ea73664 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:42.467933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c1acc-716f-441c-bce7-eaf1b56bba23 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:42.469575 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3d1c1acc-716f-441c-bce7-eaf1b56bba23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:42.497834 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c1acc-716f-441c-bce7-eaf1b56bba23 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:42.574702 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd4ef2-a081-4f7c-8f06-73213ecdae96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:42.575618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd4ef2-a081-4f7c-8f06-73213ecdae96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:42.575942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd4ef2-a081-4f7c-8f06-73213ecdae96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:42.576177 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd4ef2-a081-4f7c-8f06-73213ecdae96 admin admin] PATH_INFO: `/v3/projects/59b447c810444ef299c26863b4c70ad6` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:42.577422 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd4ef2-a081-4f7c-8f06-73213ecdae96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59b447c810444ef299c26863b4c70ad6)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:42.579385 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd4ef2-a081-4f7c-8f06-73213ecdae96 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:42.614035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:42.633403 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-63fd4ef2-a081-4f7c-8f06-73213ecdae96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59b447c810444ef299c26863b4c70ad6', 'request_id': 'req-63fd4ef2-a081-4f7c-8f06-73213ecdae96'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:42.641453 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-63fd4ef2-a081-4f7c-8f06-73213ecdae96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59b447c810444ef299c26863b4c70ad6', 'request_id': 'req-63fd4ef2-a081-4f7c-8f06-73213ecdae96'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:42.648111 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-63fd4ef2-a081-4f7c-8f06-73213ecdae96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59b447c810444ef299c26863b4c70ad6', 'request_id': 'req-63fd4ef2-a081-4f7c-8f06-73213ecdae96'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:42.661699 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-63fd4ef2-a081-4f7c-8f06-73213ecdae96 admin admin] The token cache is being invalidate because project 59b447c810444ef299c26863b4c70ad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:02:42.661895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-63fd4ef2-a081-4f7c-8f06-73213ecdae96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:42.662581 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-63fd4ef2-a081-4f7c-8f06-73213ecdae96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:42.664119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1184/2382] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:02:42 2022] DELETE /identity/v3/projects/59b447c810444ef299c26863b4c70ad6 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:02:42.669735 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdafa3-2e00-4f78-909c-504c27eefa82 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:42.670077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdafa3-2e00-4f78-909c-504c27eefa82 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:42.670296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdafa3-2e00-4f78-909c-504c27eefa82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:42.691236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-06fdafa3-2e00-4f78-909c-504c27eefa82 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:42.692447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-06fdafa3-2e00-4f78-909c-504c27eefa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:42.757389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1199/2383] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:02:42.766851 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f0d91-83d1-4f62-b212-0f93c28b1f5e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:42.767364 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f0d91-83d1-4f62-b212-0f93c28b1f5e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:42.767480 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f0d91-83d1-4f62-b212-0f93c28b1f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:42.787646 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-fe2f0d91-83d1-4f62-b212-0f93c28b1f5e None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:42.788851 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-fe2f0d91-83d1-4f62-b212-0f93c28b1f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:42.800582 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1185/2384] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:42.805065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fdafa3-2e00-4f78-909c-504c27eefa82 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:42.814957 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fdafa3-2e00-4f78-909c-504c27eefa82 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:42.819497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edcb1d-ae02-4590-a06a-4c27fd15cb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:42.820169 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edcb1d-ae02-4590-a06a-4c27fd15cb16 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:42.820436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edcb1d-ae02-4590-a06a-4c27fd15cb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:42.820709 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edcb1d-ae02-4590-a06a-4c27fd15cb16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:42.822183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edcb1d-ae02-4590-a06a-4c27fd15cb16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:02:42.822828 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edcb1d-ae02-4590-a06a-4c27fd15cb16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:42.824328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edcb1d-ae02-4590-a06a-4c27fd15cb16 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:42.837093 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1200/2385] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:02:42.841739 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f0d91-83d1-4f62-b212-0f93c28b1f5e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:42.851413 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f0d91-83d1-4f62-b212-0f93c28b1f5e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:42.857303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545b757-a725-45e9-aab4-9ae43cefef5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:42.857975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545b757-a725-45e9-aab4-9ae43cefef5d admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:42.858255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545b757-a725-45e9-aab4-9ae43cefef5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:42.858528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545b757-a725-45e9-aab4-9ae43cefef5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:42.859458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545b757-a725-45e9-aab4-9ae43cefef5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:42.861371 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545b757-a725-45e9-aab4-9ae43cefef5d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:42.892422 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1186/2386] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:02:42 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:02:42.893545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edcb1d-ae02-4590-a06a-4c27fd15cb16 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:42.906084 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edcb1d-ae02-4590-a06a-4c27fd15cb16 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:42.970285 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc893c-4221-4fe8-850a-52898b03a35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:42.971010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc893c-4221-4fe8-850a-52898b03a35b admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:42.971298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc893c-4221-4fe8-850a-52898b03a35b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:42.971575 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc893c-4221-4fe8-850a-52898b03a35b admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:42.972280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc893c-4221-4fe8-850a-52898b03a35b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:42.974212 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc893c-4221-4fe8-850a-52898b03a35b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:42.999343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1201/2387] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:02:42 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:43.000719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545b757-a725-45e9-aab4-9ae43cefef5d admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:43.021391 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545b757-a725-45e9-aab4-9ae43cefef5d admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:43.034754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea83e32-9a46-41d3-b47d-1d1ca00bf804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:43.035625 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea83e32-9a46-41d3-b47d-1d1ca00bf804 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:43.035857 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea83e32-9a46-41d3-b47d-1d1ca00bf804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:43.036183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea83e32-9a46-41d3-b47d-1d1ca00bf804 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:43.036857 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea83e32-9a46-41d3-b47d-1d1ca00bf804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:43.043507 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea83e32-9a46-41d3-b47d-1d1ca00bf804 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:43.065242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1187/2388] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:43.066545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc893c-4221-4fe8-850a-52898b03a35b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:43.080354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc893c-4221-4fe8-850a-52898b03a35b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:43.093322 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140797c1-6def-4231-b0ad-b554ef9273e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:43.094183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140797c1-6def-4231-b0ad-b554ef9273e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:43.094491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140797c1-6def-4231-b0ad-b554ef9273e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:43.095615 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140797c1-6def-4231-b0ad-b554ef9273e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:43.096033 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140797c1-6def-4231-b0ad-b554ef9273e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:43.097655 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140797c1-6def-4231-b0ad-b554ef9273e1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:43.109416 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1202/2389] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:02:43.113626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea83e32-9a46-41d3-b47d-1d1ca00bf804 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:43.125783 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea83e32-9a46-41d3-b47d-1d1ca00bf804 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:43.131547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c4636-9cd0-403b-8f65-b027f45da663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:43.132586 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c4636-9cd0-403b-8f65-b027f45da663 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:43.132928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c4636-9cd0-403b-8f65-b027f45da663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:43.133202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c4636-9cd0-403b-8f65-b027f45da663 admin admin] PATH_INFO: `/v3/projects/81d0ed09c6d84bd2806f1390ff4655cc/users/8ae0205927ba45a9afe9d72806dd6450/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:43.149520 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c4636-9cd0-403b-8f65-b027f45da663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81d0ed09c6d84bd2806f1390ff4655cc, user_id=8ae0205927ba45a9afe9d72806dd6450, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:43.153826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c4636-9cd0-403b-8f65-b027f45da663 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:43.187417 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1188/2390] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:02:43 2022] PUT /identity/v3/projects/81d0ed09c6d84bd2806f1390ff4655cc/users/8ae0205927ba45a9afe9d72806dd6450/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:43.191794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607d9f7-306c-4f99-a396-b17bd4a2fcaa None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:43.192063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607d9f7-306c-4f99-a396-b17bd4a2fcaa None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:43.192340 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607d9f7-306c-4f99-a396-b17bd4a2fcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:43.224416 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2607d9f7-306c-4f99-a396-b17bd4a2fcaa None None] MFA Rules not processed for user `8ae0205927ba45a9afe9d72806dd6450`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:43.225829 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2607d9f7-306c-4f99-a396-b17bd4a2fcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:43.255064 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:43.313127 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1203/2391] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:43.325211 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c4636-9cd0-403b-8f65-b027f45da663 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:43.327035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5d1c4636-9cd0-403b-8f65-b027f45da663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:43.402018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c4636-9cd0-403b-8f65-b027f45da663 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:43.406660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ce710-e0af-46ea-8abf-1e207eff2a98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:43.407414 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ce710-e0af-46ea-8abf-1e207eff2a98 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:43.407763 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ce710-e0af-46ea-8abf-1e207eff2a98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:43.408107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ce710-e0af-46ea-8abf-1e207eff2a98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:43.419383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ce710-e0af-46ea-8abf-1e207eff2a98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:43.421404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ce710-e0af-46ea-8abf-1e207eff2a98 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:43.433325 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1189/2392] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:53.309925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=17 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:53.426943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=67,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:53.779602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607d9f7-306c-4f99-a396-b17bd4a2fcaa None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:53.846222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607d9f7-306c-4f99-a396-b17bd4a2fcaa None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:53.850889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5dd2f-2398-434e-b03d-45d0cb3ae4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:53.851779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5dd2f-2398-434e-b03d-45d0cb3ae4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:53.852134 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5dd2f-2398-434e-b03d-45d0cb3ae4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:53.853091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5dd2f-2398-434e-b03d-45d0cb3ae4f8 admin admin] PATH_INFO: `/v3/users/8ae0205927ba45a9afe9d72806dd6450` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:53.854339 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5dd2f-2398-434e-b03d-45d0cb3ae4f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ae0205927ba45a9afe9d72806dd6450)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:53.856332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5dd2f-2398-434e-b03d-45d0cb3ae4f8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:53.905849 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cfc5dd2f-2398-434e-b03d-45d0cb3ae4f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ae0205927ba45a9afe9d72806dd6450'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:53.906527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cfc5dd2f-2398-434e-b03d-45d0cb3ae4f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ae0205927ba45a9afe9d72806dd6450'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:53.907155 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cfc5dd2f-2398-434e-b03d-45d0cb3ae4f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ae0205927ba45a9afe9d72806dd6450'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:53.914958 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cfc5dd2f-2398-434e-b03d-45d0cb3ae4f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ae0205927ba45a9afe9d72806dd6450'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:53.923313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cfc5dd2f-2398-434e-b03d-45d0cb3ae4f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ae0205927ba45a9afe9d72806dd6450'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:53.943088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1204/2393] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:02:53 2022] DELETE /identity/v3/users/8ae0205927ba45a9afe9d72806dd6450 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:53.948204 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a739309-cf57-441b-afd7-13be8740f0f5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:53.948483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a739309-cf57-441b-afd7-13be8740f0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:53.948756 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a739309-cf57-441b-afd7-13be8740f0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:53.967716 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-7a739309-cf57-441b-afd7-13be8740f0f5 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:53.968957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7a739309-cf57-441b-afd7-13be8740f0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:54.029561 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1190/2394] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:54.039321 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5dd2f-2398-434e-b03d-45d0cb3ae4f8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:54.041046 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cfc5dd2f-2398-434e-b03d-45d0cb3ae4f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:54.102759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5dd2f-2398-434e-b03d-45d0cb3ae4f8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:54.106453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecb08d-595e-40af-b242-cc74230524e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:54.107173 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecb08d-595e-40af-b242-cc74230524e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:54.107460 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecb08d-595e-40af-b242-cc74230524e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:54.107737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecb08d-595e-40af-b242-cc74230524e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:54.116764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecb08d-595e-40af-b242-cc74230524e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:54.118191 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecb08d-595e-40af-b242-cc74230524e9 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:54.127884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1205/2395] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:02:54.400075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a739309-cf57-441b-afd7-13be8740f0f5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:54.402032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7a739309-cf57-441b-afd7-13be8740f0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:54.414350 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a739309-cf57-441b-afd7-13be8740f0f5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:54.420140 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c2db1-0da6-4f6c-bd5e-a3d715feeee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:54.420575 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c2db1-0da6-4f6c-bd5e-a3d715feeee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:54.420839 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c2db1-0da6-4f6c-bd5e-a3d715feeee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:54.421118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c2db1-0da6-4f6c-bd5e-a3d715feeee2 admin admin] PATH_INFO: `/v3/projects/81d0ed09c6d84bd2806f1390ff4655cc` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:54.422311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c2db1-0da6-4f6c-bd5e-a3d715feeee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81d0ed09c6d84bd2806f1390ff4655cc)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:54.424258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c2db1-0da6-4f6c-bd5e-a3d715feeee2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:54.472340 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-150c2db1-0da6-4f6c-bd5e-a3d715feeee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81d0ed09c6d84bd2806f1390ff4655cc', 'request_id': 'req-150c2db1-0da6-4f6c-bd5e-a3d715feeee2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:54.479823 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-150c2db1-0da6-4f6c-bd5e-a3d715feeee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81d0ed09c6d84bd2806f1390ff4655cc', 'request_id': 'req-150c2db1-0da6-4f6c-bd5e-a3d715feeee2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:54.486512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-150c2db1-0da6-4f6c-bd5e-a3d715feeee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81d0ed09c6d84bd2806f1390ff4655cc', 'request_id': 'req-150c2db1-0da6-4f6c-bd5e-a3d715feeee2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:54.498256 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-150c2db1-0da6-4f6c-bd5e-a3d715feeee2 admin admin] The token cache is being invalidate because project 81d0ed09c6d84bd2806f1390ff4655cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:02:54.498550 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-150c2db1-0da6-4f6c-bd5e-a3d715feeee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:54.499208 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-150c2db1-0da6-4f6c-bd5e-a3d715feeee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:54.500744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1191/2396] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:02:54 2022] DELETE /identity/v3/projects/81d0ed09c6d84bd2806f1390ff4655cc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:57.687932 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecb08d-595e-40af-b242-cc74230524e9 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:57.689657 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-acecb08d-595e-40af-b242-cc74230524e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:57.702502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecb08d-595e-40af-b242-cc74230524e9 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:57.726782 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:57.763235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7b2f2-8783-46a4-b0ef-a0767a39e9eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:57.763985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7b2f2-8783-46a4-b0ef-a0767a39e9eb service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:57.764338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7b2f2-8783-46a4-b0ef-a0767a39e9eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:57.764681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7b2f2-8783-46a4-b0ef-a0767a39e9eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:57.766529 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f2f7b2f2-8783-46a4-b0ef-a0767a39e9eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:57.835367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7b2f2-8783-46a4-b0ef-a0767a39e9eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:57.836897 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7b2f2-8783-46a4-b0ef-a0767a39e9eb service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:57.847368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1206/2397] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:04.495154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:05.076032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920e1b7-d76d-4806-825c-78c47847c181 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:05.076316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920e1b7-d76d-4806-825c-78c47847c181 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:05.076589 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920e1b7-d76d-4806-825c-78c47847c181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:05.094938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e920e1b7-d76d-4806-825c-78c47847c181 None None] MFA Rules not processed for user `02c826fc0baa480393df4c884c9e35dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:05.096240 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e920e1b7-d76d-4806-825c-78c47847c181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:05.153714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1192/2398] 10.222.0.70 () {58 vars in 967 bytes} [Thu Jun 16 19:03:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:05.167079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7b2f2-8783-46a4-b0ef-a0767a39e9eb service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:05.168408 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f2f7b2f2-8783-46a4-b0ef-a0767a39e9eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:05.238757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7b2f2-8783-46a4-b0ef-a0767a39e9eb service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:05.243964 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145dd6c5-a5f5-48f8-a0fc-8ac78870754e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b22fe0ab0354385924923d0d443acae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:05.244696 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145dd6c5-a5f5-48f8-a0fc-8ac78870754e service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:05.244983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145dd6c5-a5f5-48f8-a0fc-8ac78870754e service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:05.245264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145dd6c5-a5f5-48f8-a0fc-8ac78870754e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:05.253347 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145dd6c5-a5f5-48f8-a0fc-8ac78870754e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:05.254970 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145dd6c5-a5f5-48f8-a0fc-8ac78870754e service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:05.263602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1207/2399] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Jun 16 19:03:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:03:15.146341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=25 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:15.260417 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=72 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:55.017994 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315ba6f-c376-4c07-b5bd-bd3a4b2a2d0a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:55.018466 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315ba6f-c376-4c07-b5bd-bd3a4b2a2d0a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:55.018603 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315ba6f-c376-4c07-b5bd-bd3a4b2a2d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:55.037043 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e315ba6f-c376-4c07-b5bd-bd3a4b2a2d0a None None] MFA Rules not processed for user `7f703f74c7ff43a291f54103e164ccdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:55.038269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e315ba6f-c376-4c07-b5bd-bd3a4b2a2d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:55.103881 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1193/2400] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:03:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:04:05.096111 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=25 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:04:10.592455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145dd6c5-a5f5-48f8-a0fc-8ac78870754e service swift] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:10.593930 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-145dd6c5-a5f5-48f8-a0fc-8ac78870754e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:10.606757 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145dd6c5-a5f5-48f8-a0fc-8ac78870754e service swift] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:10.610261 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385014a-dbf0-4ca0-b5b2-42e16daf0124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:10.610919 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385014a-dbf0-4ca0-b5b2-42e16daf0124 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:10.611201 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385014a-dbf0-4ca0-b5b2-42e16daf0124 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:10.611473 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385014a-dbf0-4ca0-b5b2-42e16daf0124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:10.613157 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d385014a-dbf0-4ca0-b5b2-42e16daf0124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:10.683656 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385014a-dbf0-4ca0-b5b2-42e16daf0124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:10.685098 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385014a-dbf0-4ca0-b5b2-42e16daf0124 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:10.702177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1208/2401] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:04:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:04:11.880149 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315ba6f-c376-4c07-b5bd-bd3a4b2a2d0a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:11.882318 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e315ba6f-c376-4c07-b5bd-bd3a4b2a2d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:11.894556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315ba6f-c376-4c07-b5bd-bd3a4b2a2d0a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:11.898865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be3baa-afab-4a76-ac81-a826e62b9e61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:11.899533 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be3baa-afab-4a76-ac81-a826e62b9e61 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:11.899823 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be3baa-afab-4a76-ac81-a826e62b9e61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:11.900366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be3baa-afab-4a76-ac81-a826e62b9e61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:11.902161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-71be3baa-afab-4a76-ac81-a826e62b9e61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:11.916861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be3baa-afab-4a76-ac81-a826e62b9e61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:11.918297 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be3baa-afab-4a76-ac81-a826e62b9e61 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:11.930096 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1194/2402] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:04:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:04:13.064178 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74707d-cdb1-482a-aa48-f23afef4ac66 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:13.064735 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74707d-cdb1-482a-aa48-f23afef4ac66 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:13.064735 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74707d-cdb1-482a-aa48-f23afef4ac66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:13.084982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3e74707d-cdb1-482a-aa48-f23afef4ac66 None None] MFA Rules not processed for user `fb08a57a393d4c8591da42cd5e92aca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:04:13.086181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3e74707d-cdb1-482a-aa48-f23afef4ac66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:13.149420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1209/2403] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:04:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:04:21.924553 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=6 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:04:23.141860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=59 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:04:28.745813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891b9a6-12d7-4420-bda0-2c4e1c53455a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:28.746337 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891b9a6-12d7-4420-bda0-2c4e1c53455a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:28.746337 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891b9a6-12d7-4420-bda0-2c4e1c53455a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:28.766300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-3891b9a6-12d7-4420-bda0-2c4e1c53455a None None] MFA Rules not processed for user `04327f3d7b844ddb88cf4a81145634c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:04:28.767631 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3891b9a6-12d7-4420-bda0-2c4e1c53455a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:28.827877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1195/2404] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:04:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:04:29.165252 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74707d-cdb1-482a-aa48-f23afef4ac66 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:29.174963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74707d-cdb1-482a-aa48-f23afef4ac66 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:29.178591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbe004-b45a-4a6a-94b9-794908c2c18d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:29.179344 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbe004-b45a-4a6a-94b9-794908c2c18d service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:29.179624 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbe004-b45a-4a6a-94b9-794908c2c18d service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:29.179899 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbe004-b45a-4a6a-94b9-794908c2c18d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:29.181664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5dcbe004-b45a-4a6a-94b9-794908c2c18d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:29.191658 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbe004-b45a-4a6a-94b9-794908c2c18d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:29.193771 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbe004-b45a-4a6a-94b9-794908c2c18d service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:29.202327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1210/2405] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:04:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:04:30.519152 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891b9a6-12d7-4420-bda0-2c4e1c53455a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:30.529683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891b9a6-12d7-4420-bda0-2c4e1c53455a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:30.533264 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef147a31-aa2d-4195-a6dd-92590de72a6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:30.533913 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef147a31-aa2d-4195-a6dd-92590de72a6f service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:30.534194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef147a31-aa2d-4195-a6dd-92590de72a6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:30.534471 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef147a31-aa2d-4195-a6dd-92590de72a6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:30.536980 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ef147a31-aa2d-4195-a6dd-92590de72a6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:30.561227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef147a31-aa2d-4195-a6dd-92590de72a6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:30.563241 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef147a31-aa2d-4195-a6dd-92590de72a6f service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:30.573674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1196/2406] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:04:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:04:39.199118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=6 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:04:40.569046 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=31 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:04:40.985442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbe004-b45a-4a6a-94b9-794908c2c18d service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:40.995822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbe004-b45a-4a6a-94b9-794908c2c18d service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:40.999628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104e504-7095-4d37-b783-bfaae0679596 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:41.000282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104e504-7095-4d37-b783-bfaae0679596 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:41.000553 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104e504-7095-4d37-b783-bfaae0679596 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:41.000827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104e504-7095-4d37-b783-bfaae0679596 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:41.002615 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5104e504-7095-4d37-b783-bfaae0679596 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:41.012854 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5104e504-7095-4d37-b783-bfaae0679596 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:41.014285 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5104e504-7095-4d37-b783-bfaae0679596 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:41.022497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1211/2407] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:04:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:04:45.288380 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef147a31-aa2d-4195-a6dd-92590de72a6f service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:45.289977 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ef147a31-aa2d-4195-a6dd-92590de72a6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:45.374467 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef147a31-aa2d-4195-a6dd-92590de72a6f service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:45.381162 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862f58e-6bde-4e84-9915-201be908c511 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '880d0299c38c4eb897c669a512e599be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:45.381960 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862f58e-6bde-4e84-9915-201be908c511 service placement] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:45.382621 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862f58e-6bde-4e84-9915-201be908c511 service placement] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:45.382621 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862f58e-6bde-4e84-9915-201be908c511 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:45.384654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0862f58e-6bde-4e84-9915-201be908c511 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:45.398235 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862f58e-6bde-4e84-9915-201be908c511 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:45.399791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862f58e-6bde-4e84-9915-201be908c511 service placement] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:45.414719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1197/2408] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:04:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:04:51.019476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=6 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:04:52.485368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104e504-7095-4d37-b783-bfaae0679596 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:52.487153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5104e504-7095-4d37-b783-bfaae0679596 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:52.505419 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104e504-7095-4d37-b783-bfaae0679596 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:52.513692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291f969-eefd-44cb-bdf7-2c9d7a245eba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:52.514845 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291f969-eefd-44cb-bdf7-2c9d7a245eba service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:52.514845 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291f969-eefd-44cb-bdf7-2c9d7a245eba service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:52.515143 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291f969-eefd-44cb-bdf7-2c9d7a245eba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:52.516775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1291f969-eefd-44cb-bdf7-2c9d7a245eba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:52.531646 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291f969-eefd-44cb-bdf7-2c9d7a245eba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:52.533245 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291f969-eefd-44cb-bdf7-2c9d7a245eba service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:52.548936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1212/2409] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:04:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:04:55.406180 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=32 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:04:57.600963 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2b28c-c551-47e2-8e45-8c6ccb089890 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:57.601451 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2b28c-c551-47e2-8e45-8c6ccb089890 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:57.601640 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2b28c-c551-47e2-8e45-8c6ccb089890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:57.619334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-acb2b28c-c551-47e2-8e45-8c6ccb089890 None None] MFA Rules not processed for user `02c826fc0baa480393df4c884c9e35dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:04:57.620966 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-acb2b28c-c551-47e2-8e45-8c6ccb089890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:57.627956 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1198/2410] 10.222.0.70 () {58 vars in 967 bytes} [Thu Jun 16 19:04:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:04:57.677966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291f969-eefd-44cb-bdf7-2c9d7a245eba service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:57.689508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291f969-eefd-44cb-bdf7-2c9d7a245eba service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:57.695062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0644b-94ad-4bd4-ba2a-e56bfeaf675c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b22fe0ab0354385924923d0d443acae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:57.695705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0644b-94ad-4bd4-ba2a-e56bfeaf675c service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:57.695991 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0644b-94ad-4bd4-ba2a-e56bfeaf675c service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:57.696272 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0644b-94ad-4bd4-ba2a-e56bfeaf675c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:57.704370 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0644b-94ad-4bd4-ba2a-e56bfeaf675c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:57.705987 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0644b-94ad-4bd4-ba2a-e56bfeaf675c service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:57.715204 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1213/2411] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Jun 16 19:04:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:05:01.704344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2b28c-c551-47e2-8e45-8c6ccb089890 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:01.705800 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-acb2b28c-c551-47e2-8e45-8c6ccb089890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:01.765810 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2b28c-c551-47e2-8e45-8c6ccb089890 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:01.770404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2deaa-7f6b-42b7-80d0-0b272999fdc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:01.771086 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2deaa-7f6b-42b7-80d0-0b272999fdc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:01.771368 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2deaa-7f6b-42b7-80d0-0b272999fdc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:01.771650 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2deaa-7f6b-42b7-80d0-0b272999fdc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:01.773513 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7ff2deaa-7f6b-42b7-80d0-0b272999fdc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:01.791630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff2deaa-7f6b-42b7-80d0-0b272999fdc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:01.793070 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff2deaa-7f6b-42b7-80d0-0b272999fdc9 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:01.802695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1199/2412] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:05:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:05:05.067905 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0644b-94ad-4bd4-ba2a-e56bfeaf675c service swift] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:05.071334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-31f0644b-94ad-4bd4-ba2a-e56bfeaf675c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:05.177627 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0644b-94ad-4bd4-ba2a-e56bfeaf675c service swift] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:05.187111 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787754c4-e8aa-4dcf-90e2-87da773c5b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:05.188134 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787754c4-e8aa-4dcf-90e2-87da773c5b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:05.188421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787754c4-e8aa-4dcf-90e2-87da773c5b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:05.188729 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787754c4-e8aa-4dcf-90e2-87da773c5b8e admin admin] PATH_INFO: `/v3/users/047e841cefdd4975947b2c154412953b` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:05.193175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787754c4-e8aa-4dcf-90e2-87da773c5b8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=047e841cefdd4975947b2c154412953b)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:05.195430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787754c4-e8aa-4dcf-90e2-87da773c5b8e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:05.289902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-787754c4-e8aa-4dcf-90e2-87da773c5b8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '047e841cefdd4975947b2c154412953b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:05.290870 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-787754c4-e8aa-4dcf-90e2-87da773c5b8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '047e841cefdd4975947b2c154412953b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:05.294711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-787754c4-e8aa-4dcf-90e2-87da773c5b8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '047e841cefdd4975947b2c154412953b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:05.317149 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-787754c4-e8aa-4dcf-90e2-87da773c5b8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '047e841cefdd4975947b2c154412953b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:05.325020 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-787754c4-e8aa-4dcf-90e2-87da773c5b8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '047e841cefdd4975947b2c154412953b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:05.349679 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1214/2413] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:05:05 2022] DELETE /identity/v3/users/047e841cefdd4975947b2c154412953b => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:05:05.499178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2deaa-7f6b-42b7-80d0-0b272999fdc9 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:05.500630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7ff2deaa-7f6b-42b7-80d0-0b272999fdc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:05.522839 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2deaa-7f6b-42b7-80d0-0b272999fdc9 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:05.546007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787754c4-e8aa-4dcf-90e2-87da773c5b8e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:05.547547 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-787754c4-e8aa-4dcf-90e2-87da773c5b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:05.614602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:05:05.616635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fdf9b-2860-47a6-8c72-922fd92fbc0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:05.617285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fdf9b-2860-47a6-8c72-922fd92fbc0a service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:05.617615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fdf9b-2860-47a6-8c72-922fd92fbc0a service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:05.617837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fdf9b-2860-47a6-8c72-922fd92fbc0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:05.619991 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-004fdf9b-2860-47a6-8c72-922fd92fbc0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:05.627744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787754c4-e8aa-4dcf-90e2-87da773c5b8e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:05.634260 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004fdf9b-2860-47a6-8c72-922fd92fbc0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:05.634947 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716527eb-2545-490a-a860-f37779b3245e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:05.635812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716527eb-2545-490a-a860-f37779b3245e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:05.635969 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004fdf9b-2860-47a6-8c72-922fd92fbc0a service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:05.636095 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716527eb-2545-490a-a860-f37779b3245e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:05.636360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716527eb-2545-490a-a860-f37779b3245e admin admin] PATH_INFO: `/v3/projects/741ee13164894b49be1a0fc6c1385a8e` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:05.637573 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716527eb-2545-490a-a860-f37779b3245e admin admin] RBAC: Authorizing `identity:delete_project(project_id=741ee13164894b49be1a0fc6c1385a8e)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:05.639508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716527eb-2545-490a-a860-f37779b3245e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:05.647576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1200/2414] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:05:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:05:05.691555 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-716527eb-2545-490a-a860-f37779b3245e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '741ee13164894b49be1a0fc6c1385a8e', 'request_id': 'req-716527eb-2545-490a-a860-f37779b3245e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:05.699066 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-716527eb-2545-490a-a860-f37779b3245e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '741ee13164894b49be1a0fc6c1385a8e', 'request_id': 'req-716527eb-2545-490a-a860-f37779b3245e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:05.705583 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-716527eb-2545-490a-a860-f37779b3245e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '741ee13164894b49be1a0fc6c1385a8e', 'request_id': 'req-716527eb-2545-490a-a860-f37779b3245e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:05.716382 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-716527eb-2545-490a-a860-f37779b3245e admin admin] The token cache is being invalidate because project 741ee13164894b49be1a0fc6c1385a8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:05:05.716666 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-716527eb-2545-490a-a860-f37779b3245e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:05.717276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-716527eb-2545-490a-a860-f37779b3245e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:05.718863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1215/2415] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:05:05 2022] DELETE /identity/v3/projects/741ee13164894b49be1a0fc6c1385a8e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:05:05.723777 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fdf9b-2860-47a6-8c72-922fd92fbc0a service nova] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:05.734683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fdf9b-2860-47a6-8c72-922fd92fbc0a service nova] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:05.791919 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbb5bb-b74b-436b-bd7d-69fa1f5cfc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:05.792821 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbb5bb-b74b-436b-bd7d-69fa1f5cfc46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:05.793107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbb5bb-b74b-436b-bd7d-69fa1f5cfc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:05.793385 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbb5bb-b74b-436b-bd7d-69fa1f5cfc46 admin admin] PATH_INFO: `/v3/users/0fd50fda92344a1d81c3eefd8e2f2508` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:05.794618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efbb5bb-b74b-436b-bd7d-69fa1f5cfc46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fd50fda92344a1d81c3eefd8e2f2508)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:05.796535 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efbb5bb-b74b-436b-bd7d-69fa1f5cfc46 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:05.821517 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:05:05.845257 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6efbb5bb-b74b-436b-bd7d-69fa1f5cfc46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fd50fda92344a1d81c3eefd8e2f2508'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:05.845919 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6efbb5bb-b74b-436b-bd7d-69fa1f5cfc46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fd50fda92344a1d81c3eefd8e2f2508'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:05.853488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6efbb5bb-b74b-436b-bd7d-69fa1f5cfc46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fd50fda92344a1d81c3eefd8e2f2508'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:05.866734 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6efbb5bb-b74b-436b-bd7d-69fa1f5cfc46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fd50fda92344a1d81c3eefd8e2f2508'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:05.867327 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6efbb5bb-b74b-436b-bd7d-69fa1f5cfc46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fd50fda92344a1d81c3eefd8e2f2508'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:05.881124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1201/2416] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:05:05 2022] DELETE /identity/v3/users/0fd50fda92344a1d81c3eefd8e2f2508 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:05:06.065880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716527eb-2545-490a-a860-f37779b3245e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:06.067803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-716527eb-2545-490a-a860-f37779b3245e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:06.087633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716527eb-2545-490a-a860-f37779b3245e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:06.147566 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e8654-1ad8-4bcd-8134-75344d4dc49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:06.148354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e8654-1ad8-4bcd-8134-75344d4dc49b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:06.148640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e8654-1ad8-4bcd-8134-75344d4dc49b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:06.148917 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e8654-1ad8-4bcd-8134-75344d4dc49b admin admin] PATH_INFO: `/v3/projects/4eafa3e811a6497faa943075af273826` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:06.150378 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e8654-1ad8-4bcd-8134-75344d4dc49b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eafa3e811a6497faa943075af273826)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:06.152405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e8654-1ad8-4bcd-8134-75344d4dc49b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:06.199915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3d6e8654-1ad8-4bcd-8134-75344d4dc49b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eafa3e811a6497faa943075af273826', 'request_id': 'req-3d6e8654-1ad8-4bcd-8134-75344d4dc49b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:06.208290 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3d6e8654-1ad8-4bcd-8134-75344d4dc49b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eafa3e811a6497faa943075af273826', 'request_id': 'req-3d6e8654-1ad8-4bcd-8134-75344d4dc49b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:06.214586 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3d6e8654-1ad8-4bcd-8134-75344d4dc49b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eafa3e811a6497faa943075af273826', 'request_id': 'req-3d6e8654-1ad8-4bcd-8134-75344d4dc49b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:06.225359 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3d6e8654-1ad8-4bcd-8134-75344d4dc49b admin admin] The token cache is being invalidate because project 4eafa3e811a6497faa943075af273826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:05:06.225647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3d6e8654-1ad8-4bcd-8134-75344d4dc49b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:06.226282 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3d6e8654-1ad8-4bcd-8134-75344d4dc49b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:06.227848 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1216/2417] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:05:06 2022] DELETE /identity/v3/projects/4eafa3e811a6497faa943075af273826 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:05:09.743358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbb5bb-b74b-436b-bd7d-69fa1f5cfc46 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:09.744837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6efbb5bb-b74b-436b-bd7d-69fa1f5cfc46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:09.807919 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbb5bb-b74b-436b-bd7d-69fa1f5cfc46 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:09.865809 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492977a-672e-46e1-b3c8-75a6226a1220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:09.866467 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492977a-672e-46e1-b3c8-75a6226a1220 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:09.866781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492977a-672e-46e1-b3c8-75a6226a1220 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:09.867064 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492977a-672e-46e1-b3c8-75a6226a1220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:09.868940 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9492977a-672e-46e1-b3c8-75a6226a1220 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:09.935227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9492977a-672e-46e1-b3c8-75a6226a1220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:09.936707 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9492977a-672e-46e1-b3c8-75a6226a1220 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:09.951289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1202/2418] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:05:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:05:16.221848 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=78,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:05:19.389556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea34f5-44f0-41c9-859e-886d227cb20b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:19.390152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea34f5-44f0-41c9-859e-886d227cb20b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:19.390152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea34f5-44f0-41c9-859e-886d227cb20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:19.411660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-34ea34f5-44f0-41c9-859e-886d227cb20b None None] MFA Rules not processed for user `8742a73e9b4f46deb0b5c9e1756ffe2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:05:19.412858 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-34ea34f5-44f0-41c9-859e-886d227cb20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:19.421292 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1217/2419] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:05:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:05:19.942133 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=88,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:05:21.040851 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492977a-672e-46e1-b3c8-75a6226a1220 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:21.042413 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9492977a-672e-46e1-b3c8-75a6226a1220 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:21.109009 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492977a-672e-46e1-b3c8-75a6226a1220 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:21.113724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188de62-d334-43e4-ac80-f58e08c94f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:21.114607 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188de62-d334-43e4-ac80-f58e08c94f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:21.114904 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188de62-d334-43e4-ac80-f58e08c94f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:21.115179 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188de62-d334-43e4-ac80-f58e08c94f90 admin admin] PATH_INFO: `/v3/users/fb08a57a393d4c8591da42cd5e92aca9` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:21.116357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188de62-d334-43e4-ac80-f58e08c94f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb08a57a393d4c8591da42cd5e92aca9)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:21.118265 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188de62-d334-43e4-ac80-f58e08c94f90 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:21.168471 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3188de62-d334-43e4-ac80-f58e08c94f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb08a57a393d4c8591da42cd5e92aca9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:21.169191 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3188de62-d334-43e4-ac80-f58e08c94f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb08a57a393d4c8591da42cd5e92aca9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:21.176687 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3188de62-d334-43e4-ac80-f58e08c94f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb08a57a393d4c8591da42cd5e92aca9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:21.189474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3188de62-d334-43e4-ac80-f58e08c94f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb08a57a393d4c8591da42cd5e92aca9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:21.190513 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3188de62-d334-43e4-ac80-f58e08c94f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb08a57a393d4c8591da42cd5e92aca9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:21.204229 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1203/2420] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:05:21 2022] DELETE /identity/v3/users/fb08a57a393d4c8591da42cd5e92aca9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:05:21.386793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea34f5-44f0-41c9-859e-886d227cb20b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:21.398887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea34f5-44f0-41c9-859e-886d227cb20b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:21.456424 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061d3ff-855d-4b2b-88ba-3afd50f3366e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:21.457287 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061d3ff-855d-4b2b-88ba-3afd50f3366e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:21.457564 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061d3ff-855d-4b2b-88ba-3afd50f3366e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:21.457835 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061d3ff-855d-4b2b-88ba-3afd50f3366e admin admin] PATH_INFO: `/v3/projects/ddb2c879d7f44189ab76e377c33f5e35` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:21.459064 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061d3ff-855d-4b2b-88ba-3afd50f3366e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddb2c879d7f44189ab76e377c33f5e35)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:21.460959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061d3ff-855d-4b2b-88ba-3afd50f3366e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:21.506685 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8061d3ff-855d-4b2b-88ba-3afd50f3366e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddb2c879d7f44189ab76e377c33f5e35', 'request_id': 'req-8061d3ff-855d-4b2b-88ba-3afd50f3366e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:21.514048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8061d3ff-855d-4b2b-88ba-3afd50f3366e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddb2c879d7f44189ab76e377c33f5e35', 'request_id': 'req-8061d3ff-855d-4b2b-88ba-3afd50f3366e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:21.521224 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8061d3ff-855d-4b2b-88ba-3afd50f3366e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddb2c879d7f44189ab76e377c33f5e35', 'request_id': 'req-8061d3ff-855d-4b2b-88ba-3afd50f3366e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:21.531937 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8061d3ff-855d-4b2b-88ba-3afd50f3366e admin admin] The token cache is being invalidate because project ddb2c879d7f44189ab76e377c33f5e35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:05:21.532270 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8061d3ff-855d-4b2b-88ba-3afd50f3366e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:21.532904 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8061d3ff-855d-4b2b-88ba-3afd50f3366e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:21.534656 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1218/2421] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:05:21 2022] DELETE /identity/v3/projects/ddb2c879d7f44189ab76e377c33f5e35 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:05:25.092716 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188de62-d334-43e4-ac80-f58e08c94f90 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:25.094440 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3188de62-d334-43e4-ac80-f58e08c94f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:25.170321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188de62-d334-43e4-ac80-f58e08c94f90 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:25.230100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:05:25.273607 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd93229-3877-4150-80e3-744a969f6b69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:25.275028 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd93229-3877-4150-80e3-744a969f6b69 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:25.275476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd93229-3877-4150-80e3-744a969f6b69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:25.275768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd93229-3877-4150-80e3-744a969f6b69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:25.277596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ddd93229-3877-4150-80e3-744a969f6b69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:25.359093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd93229-3877-4150-80e3-744a969f6b69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:25.360574 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd93229-3877-4150-80e3-744a969f6b69 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:25.372872 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1204/2422] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:05:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:05:28.920102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061d3ff-855d-4b2b-88ba-3afd50f3366e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:28.921646 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8061d3ff-855d-4b2b-88ba-3afd50f3366e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:28.936497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061d3ff-855d-4b2b-88ba-3afd50f3366e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:28.941360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd1924-b1dd-4282-a893-6c55a26aba3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:28.942288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd1924-b1dd-4282-a893-6c55a26aba3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:28.942585 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd1924-b1dd-4282-a893-6c55a26aba3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:28.942910 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd1924-b1dd-4282-a893-6c55a26aba3f admin admin] PATH_INFO: `/v3/users/7f703f74c7ff43a291f54103e164ccdf` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:28.944124 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd1924-b1dd-4282-a893-6c55a26aba3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f703f74c7ff43a291f54103e164ccdf)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:28.946056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd1924-b1dd-4282-a893-6c55a26aba3f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:28.994973 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e8fd1924-b1dd-4282-a893-6c55a26aba3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f703f74c7ff43a291f54103e164ccdf'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:28.995619 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e8fd1924-b1dd-4282-a893-6c55a26aba3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f703f74c7ff43a291f54103e164ccdf'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:28.996219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e8fd1924-b1dd-4282-a893-6c55a26aba3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f703f74c7ff43a291f54103e164ccdf'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:29.003681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e8fd1924-b1dd-4282-a893-6c55a26aba3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f703f74c7ff43a291f54103e164ccdf'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:29.010926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e8fd1924-b1dd-4282-a893-6c55a26aba3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f703f74c7ff43a291f54103e164ccdf'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:29.022262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:05:29.030271 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1219/2423] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:05:28 2022] DELETE /identity/v3/users/7f703f74c7ff43a291f54103e164ccdf => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:05:29.213293 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd93229-3877-4150-80e3-744a969f6b69 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:29.224977 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd93229-3877-4150-80e3-744a969f6b69 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:29.301316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72aeb2-9a24-4662-8ee8-661a1cba46f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:29.302090 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72aeb2-9a24-4662-8ee8-661a1cba46f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:29.302480 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72aeb2-9a24-4662-8ee8-661a1cba46f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:29.302888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72aeb2-9a24-4662-8ee8-661a1cba46f8 admin admin] PATH_INFO: `/v3/projects/83b304be7bd84301b898bb50f4ac7ae4` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:29.304508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72aeb2-9a24-4662-8ee8-661a1cba46f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83b304be7bd84301b898bb50f4ac7ae4)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:29.307141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72aeb2-9a24-4662-8ee8-661a1cba46f8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:29.360922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cc72aeb2-9a24-4662-8ee8-661a1cba46f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83b304be7bd84301b898bb50f4ac7ae4', 'request_id': 'req-cc72aeb2-9a24-4662-8ee8-661a1cba46f8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:29.371630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cc72aeb2-9a24-4662-8ee8-661a1cba46f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83b304be7bd84301b898bb50f4ac7ae4', 'request_id': 'req-cc72aeb2-9a24-4662-8ee8-661a1cba46f8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:29.378456 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cc72aeb2-9a24-4662-8ee8-661a1cba46f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83b304be7bd84301b898bb50f4ac7ae4', 'request_id': 'req-cc72aeb2-9a24-4662-8ee8-661a1cba46f8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:29.390286 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cc72aeb2-9a24-4662-8ee8-661a1cba46f8 admin admin] The token cache is being invalidate because project 83b304be7bd84301b898bb50f4ac7ae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:05:29.390643 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cc72aeb2-9a24-4662-8ee8-661a1cba46f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:29.391483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cc72aeb2-9a24-4662-8ee8-661a1cba46f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:05:29.393830 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1205/2424] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:05:29 2022] DELETE /identity/v3/projects/83b304be7bd84301b898bb50f4ac7ae4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:05:29.398734 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8751ef-e1b2-413d-ac7c-5b87d8a81a13 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:29.399029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8751ef-e1b2-413d-ac7c-5b87d8a81a13 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:29.399312 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8751ef-e1b2-413d-ac7c-5b87d8a81a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:29.419220 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-da8751ef-e1b2-413d-ac7c-5b87d8a81a13 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:05:29.420437 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-da8751ef-e1b2-413d-ac7c-5b87d8a81a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:29.497342 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1220/2425] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:05:29.506591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d068647-547b-499b-a378-95087a50c9f7 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:29.507586 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d068647-547b-499b-a378-95087a50c9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:29.508934 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d068647-547b-499b-a378-95087a50c9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:29.530080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-9d068647-547b-499b-a378-95087a50c9f7 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:05:29.531352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9d068647-547b-499b-a378-95087a50c9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:29.542937 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1206/2426] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:05:29.547363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8751ef-e1b2-413d-ac7c-5b87d8a81a13 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:29.566234 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8751ef-e1b2-413d-ac7c-5b87d8a81a13 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:29.571337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d074f-76eb-4edd-bcfb-916acd85537e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:29.572025 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d074f-76eb-4edd-bcfb-916acd85537e admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:29.572998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d074f-76eb-4edd-bcfb-916acd85537e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:29.573283 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d074f-76eb-4edd-bcfb-916acd85537e admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:29.574041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d074f-76eb-4edd-bcfb-916acd85537e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:05:29.574528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d074f-76eb-4edd-bcfb-916acd85537e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:29.576243 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d074f-76eb-4edd-bcfb-916acd85537e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:29.592360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1221/2427] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:05:29.596704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d068647-547b-499b-a378-95087a50c9f7 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:29.608287 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d068647-547b-499b-a378-95087a50c9f7 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:29.617452 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c463e-5740-47a6-92f0-cb5e9db92d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:29.618282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c463e-5740-47a6-92f0-cb5e9db92d25 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:29.618727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c463e-5740-47a6-92f0-cb5e9db92d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:29.619105 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c463e-5740-47a6-92f0-cb5e9db92d25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:29.620108 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c463e-5740-47a6-92f0-cb5e9db92d25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:29.628252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c463e-5740-47a6-92f0-cb5e9db92d25 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:29.633751 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:05:29.683748 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1207/2428] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:05:29 2022] POST /identity/v3/projects => generated 371 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:05:29.684914 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d074f-76eb-4edd-bcfb-916acd85537e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:29.696929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d074f-76eb-4edd-bcfb-916acd85537e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:29.771181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83d72f-2913-4fd6-b56f-a40fa2a19e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:29.772044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83d72f-2913-4fd6-b56f-a40fa2a19e0f admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:29.772395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83d72f-2913-4fd6-b56f-a40fa2a19e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:29.772732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83d72f-2913-4fd6-b56f-a40fa2a19e0f admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:29.773510 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83d72f-2913-4fd6-b56f-a40fa2a19e0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:29.775842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83d72f-2913-4fd6-b56f-a40fa2a19e0f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:29.797544 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1222/2429] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:05:29 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:05:29.802101 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c463e-5740-47a6-92f0-cb5e9db92d25 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:29.814069 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c463e-5740-47a6-92f0-cb5e9db92d25 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:29.819070 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2117f-a14a-430b-b62a-96bc541591aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:29.819836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2117f-a14a-430b-b62a-96bc541591aa admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:29.820198 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2117f-a14a-430b-b62a-96bc541591aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:29.820545 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2117f-a14a-430b-b62a-96bc541591aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:29.821380 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2117f-a14a-430b-b62a-96bc541591aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:29.823039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2117f-a14a-430b-b62a-96bc541591aa admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:29.834060 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1208/2430] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:05:29.838040 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83d72f-2913-4fd6-b56f-a40fa2a19e0f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:29.848950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83d72f-2913-4fd6-b56f-a40fa2a19e0f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:29.853623 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b1e10-354e-46f3-94a6-c06907a954c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:29.854288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b1e10-354e-46f3-94a6-c06907a954c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:29.854590 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b1e10-354e-46f3-94a6-c06907a954c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:29.854877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b1e10-354e-46f3-94a6-c06907a954c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:29.855602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b1e10-354e-46f3-94a6-c06907a954c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:29.857202 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b1e10-354e-46f3-94a6-c06907a954c7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:29.868395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1223/2431] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:05:29.872417 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2117f-a14a-430b-b62a-96bc541591aa admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:29.882526 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2117f-a14a-430b-b62a-96bc541591aa admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:29.887320 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9576d4-6995-4889-9bf3-8627728eda3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:29.888188 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9576d4-6995-4889-9bf3-8627728eda3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:29.888469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9576d4-6995-4889-9bf3-8627728eda3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:29.888745 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9576d4-6995-4889-9bf3-8627728eda3a admin admin] PATH_INFO: `/v3/projects/17adf3d56ff04eef9e5bc04252cc0c43/users/72275ed9c4594ca5b22a44cecfb6151d/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:29.903538 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9576d4-6995-4889-9bf3-8627728eda3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=17adf3d56ff04eef9e5bc04252cc0c43, user_id=72275ed9c4594ca5b22a44cecfb6151d, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:29.906181 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9576d4-6995-4889-9bf3-8627728eda3a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:29.920538 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1209/2432] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:05:29 2022] PUT /identity/v3/projects/17adf3d56ff04eef9e5bc04252cc0c43/users/72275ed9c4594ca5b22a44cecfb6151d/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:05:29.931295 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182008f4-2a0d-4b27-a23d-2cc0cbbb96b9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:29.931839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182008f4-2a0d-4b27-a23d-2cc0cbbb96b9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:29.931839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182008f4-2a0d-4b27-a23d-2cc0cbbb96b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:29.961629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-182008f4-2a0d-4b27-a23d-2cc0cbbb96b9 None None] MFA Rules not processed for user `72275ed9c4594ca5b22a44cecfb6151d`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:05:29.962997 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-182008f4-2a0d-4b27-a23d-2cc0cbbb96b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:30.012425 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:05:30.031527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1224/2433] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:05:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:05:30.037267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9576d4-6995-4889-9bf3-8627728eda3a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:30.103570 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9576d4-6995-4889-9bf3-8627728eda3a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:30.108430 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02cd58-1835-4fae-a602-730a743229be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:30.109092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02cd58-1835-4fae-a602-730a743229be admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:30.109370 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02cd58-1835-4fae-a602-730a743229be admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:30.109644 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02cd58-1835-4fae-a602-730a743229be admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:30.110342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02cd58-1835-4fae-a602-730a743229be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:30.112943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02cd58-1835-4fae-a602-730a743229be admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:30.138831 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1210/2434] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:05:30 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:05:30.143243 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182008f4-2a0d-4b27-a23d-2cc0cbbb96b9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:30.155066 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182008f4-2a0d-4b27-a23d-2cc0cbbb96b9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:30.212441 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f08ef6-0f4b-4356-bc15-8ea0f7912c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:30.213238 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f08ef6-0f4b-4356-bc15-8ea0f7912c50 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:30.213534 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f08ef6-0f4b-4356-bc15-8ea0f7912c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:30.213815 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f08ef6-0f4b-4356-bc15-8ea0f7912c50 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:30.214512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f08ef6-0f4b-4356-bc15-8ea0f7912c50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:30.216468 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f08ef6-0f4b-4356-bc15-8ea0f7912c50 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:30.237130 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1225/2435] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:05:30 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:05:30.241363 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02cd58-1835-4fae-a602-730a743229be admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:30.252670 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02cd58-1835-4fae-a602-730a743229be admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:30.257351 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2ad56-2413-4e89-a9eb-959242b22465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:30.258009 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2ad56-2413-4e89-a9eb-959242b22465 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:30.258311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2ad56-2413-4e89-a9eb-959242b22465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:30.258641 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2ad56-2413-4e89-a9eb-959242b22465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:30.259367 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2ad56-2413-4e89-a9eb-959242b22465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:30.260958 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2ad56-2413-4e89-a9eb-959242b22465 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:30.271753 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1211/2436] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:05:30.276171 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f08ef6-0f4b-4356-bc15-8ea0f7912c50 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:30.286358 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f08ef6-0f4b-4356-bc15-8ea0f7912c50 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:30.291031 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6042f-3f6f-4fca-90c0-fdd1f39f626f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:30.291960 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6042f-3f6f-4fca-90c0-fdd1f39f626f admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:30.292248 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6042f-3f6f-4fca-90c0-fdd1f39f626f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:30.292525 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6042f-3f6f-4fca-90c0-fdd1f39f626f admin admin] PATH_INFO: `/v3/projects/445a3cec63f5411baf11ebfd69e2683a/users/0b1914ecb4944f1d8d8983ba3dd9a57d/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:30.308267 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6042f-3f6f-4fca-90c0-fdd1f39f626f admin admin] RBAC: Authorizing `identity:create_grant(project_id=445a3cec63f5411baf11ebfd69e2683a, user_id=0b1914ecb4944f1d8d8983ba3dd9a57d, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:30.311014 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6042f-3f6f-4fca-90c0-fdd1f39f626f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:30.324990 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1226/2437] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:05:30 2022] PUT /identity/v3/projects/445a3cec63f5411baf11ebfd69e2683a/users/0b1914ecb4944f1d8d8983ba3dd9a57d/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:05:30.328998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2ad56-2413-4e89-a9eb-959242b22465 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:30.339535 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2ad56-2413-4e89-a9eb-959242b22465 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:30.397157 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826f706-7039-4e6d-844f-3c49fc0c4f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:30.397824 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826f706-7039-4e6d-844f-3c49fc0c4f72 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:30.398107 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826f706-7039-4e6d-844f-3c49fc0c4f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:30.398384 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826f706-7039-4e6d-844f-3c49fc0c4f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:30.399130 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826f706-7039-4e6d-844f-3c49fc0c4f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:30.401005 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826f706-7039-4e6d-844f-3c49fc0c4f72 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:30.412095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1212/2438] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:05:30.416251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6042f-3f6f-4fca-90c0-fdd1f39f626f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:30.427382 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6042f-3f6f-4fca-90c0-fdd1f39f626f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:30.432001 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6b361-3b97-48e5-8685-51f7df5315b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:30.432902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6b361-3b97-48e5-8685-51f7df5315b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:30.433182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6b361-3b97-48e5-8685-51f7df5315b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:30.433449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6b361-3b97-48e5-8685-51f7df5315b8 admin admin] PATH_INFO: `/v3/projects/445a3cec63f5411baf11ebfd69e2683a/users/0b1914ecb4944f1d8d8983ba3dd9a57d/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:30.435439 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6b361-3b97-48e5-8685-51f7df5315b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=445a3cec63f5411baf11ebfd69e2683a, user_id=0b1914ecb4944f1d8d8983ba3dd9a57d, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:30.438065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6b361-3b97-48e5-8685-51f7df5315b8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:30.448767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1227/2439] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:05:30 2022] PUT /identity/v3/projects/445a3cec63f5411baf11ebfd69e2683a/users/0b1914ecb4944f1d8d8983ba3dd9a57d/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:05:30.460816 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35b0ad-2041-4bb3-9ec0-275982a0c242 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:30.461102 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35b0ad-2041-4bb3-9ec0-275982a0c242 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:30.461381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35b0ad-2041-4bb3-9ec0-275982a0c242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:30.490270 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:05:30.494047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-6d35b0ad-2041-4bb3-9ec0-275982a0c242 None None] MFA Rules not processed for user `0b1914ecb4944f1d8d8983ba3dd9a57d`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:05:30.496197 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6d35b0ad-2041-4bb3-9ec0-275982a0c242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:30.567963 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1213/2440] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:05:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:05:30.579157 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6b361-3b97-48e5-8685-51f7df5315b8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:30.581093 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bfb6b361-3b97-48e5-8685-51f7df5315b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:30.639286 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6b361-3b97-48e5-8685-51f7df5315b8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:30.642874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750d68d-75c5-47fb-8c99-58440a66608c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:30.643505 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750d68d-75c5-47fb-8c99-58440a66608c service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:30.643786 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750d68d-75c5-47fb-8c99-58440a66608c service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:30.644061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750d68d-75c5-47fb-8c99-58440a66608c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:30.653371 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750d68d-75c5-47fb-8c99-58440a66608c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:30.655370 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750d68d-75c5-47fb-8c99-58440a66608c service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:30.665989 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1228/2441] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:05:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:05:30.741678 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35b0ad-2041-4bb3-9ec0-275982a0c242 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:30.751289 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35b0ad-2041-4bb3-9ec0-275982a0c242 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:30.755762 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e9517-20f4-4305-a949-1bcc59562c2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:30.756473 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e9517-20f4-4305-a949-1bcc59562c2b service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:30.756831 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e9517-20f4-4305-a949-1bcc59562c2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:30.757184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e9517-20f4-4305-a949-1bcc59562c2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:30.819917 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e9517-20f4-4305-a949-1bcc59562c2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:30.821368 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e9517-20f4-4305-a949-1bcc59562c2b service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:30.831404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1214/2442] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:05:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:05:40.661401 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=74,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:05:40.826259 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=56 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:05:41.241165 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97717bcf-5b56-4d1a-b785-f81596f92391 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:41.241584 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97717bcf-5b56-4d1a-b785-f81596f92391 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:41.241959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97717bcf-5b56-4d1a-b785-f81596f92391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:41.264344 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-97717bcf-5b56-4d1a-b785-f81596f92391 None None] MFA Rules not processed for user `922a970a740444a2959c6aa1d0921ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:05:41.266137 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-97717bcf-5b56-4d1a-b785-f81596f92391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:41.331798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1229/2443] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:05:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:05:41.647398 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e9517-20f4-4305-a949-1bcc59562c2b service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:05:41.648883 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0c9e9517-20f4-4305-a949-1bcc59562c2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:41.707031 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e9517-20f4-4305-a949-1bcc59562c2b service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:05:41.710685 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3187671-267f-4d0d-bdda-2c8d5c78adf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:05:41.711349 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3187671-267f-4d0d-bdda-2c8d5c78adf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:41.711636 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3187671-267f-4d0d-bdda-2c8d5c78adf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:41.711917 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3187671-267f-4d0d-bdda-2c8d5c78adf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:41.713672 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a3187671-267f-4d0d-bdda-2c8d5c78adf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:41.723814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3187671-267f-4d0d-bdda-2c8d5c78adf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:05:41.725250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3187671-267f-4d0d-bdda-2c8d5c78adf2 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:05:41.733223 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1215/2444] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:05:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:05:51.324688 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=25 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:05:51.729655 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=26 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:04.630929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97717bcf-5b56-4d1a-b785-f81596f92391 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:04.632370 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-97717bcf-5b56-4d1a-b785-f81596f92391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:04.709018 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97717bcf-5b56-4d1a-b785-f81596f92391 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:04.714342 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b71513-2138-4cbd-a448-58b5b2950327 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:04.715012 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b71513-2138-4cbd-a448-58b5b2950327 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:04.715294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b71513-2138-4cbd-a448-58b5b2950327 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:04.715568 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b71513-2138-4cbd-a448-58b5b2950327 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:04.717335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-87b71513-2138-4cbd-a448-58b5b2950327 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:04.729887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b71513-2138-4cbd-a448-58b5b2950327 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:04.731487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b71513-2138-4cbd-a448-58b5b2950327 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:04.742893 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1230/2445] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:09.550090 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a195de-21c3-4f10-9d4a-60b12673c563 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:09.550646 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a195de-21c3-4f10-9d4a-60b12673c563 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:09.550791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a195de-21c3-4f10-9d4a-60b12673c563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:09.567982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-52a195de-21c3-4f10-9d4a-60b12673c563 None None] MFA Rules not processed for user `02c826fc0baa480393df4c884c9e35dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:09.569184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-52a195de-21c3-4f10-9d4a-60b12673c563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:09.627473 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1216/2446] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Jun 16 19:06:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:09.686037 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b71513-2138-4cbd-a448-58b5b2950327 service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:09.686037 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-87b71513-2138-4cbd-a448-58b5b2950327 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:09.770305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b71513-2138-4cbd-a448-58b5b2950327 service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:09.778602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbf369-24b0-4511-a3eb-a190a02d765e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b22fe0ab0354385924923d0d443acae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:09.779262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbf369-24b0-4511-a3eb-a190a02d765e service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:09.779548 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbf369-24b0-4511-a3eb-a190a02d765e service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:09.779821 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbf369-24b0-4511-a3eb-a190a02d765e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:09.790356 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbf369-24b0-4511-a3eb-a190a02d765e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:09.792017 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbf369-24b0-4511-a3eb-a190a02d765e service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:09.802381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1231/2447] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Jun 16 19:06:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:06:12.895735 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a195de-21c3-4f10-9d4a-60b12673c563 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:12.977166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a195de-21c3-4f10-9d4a-60b12673c563 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:12.982822 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14572e29-f895-4b1f-b676-2c039bd1a241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:12.983815 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14572e29-f895-4b1f-b676-2c039bd1a241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:12.984103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14572e29-f895-4b1f-b676-2c039bd1a241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:12.984912 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14572e29-f895-4b1f-b676-2c039bd1a241 admin admin] PATH_INFO: `/v3/users/72275ed9c4594ca5b22a44cecfb6151d` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:12.986388 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14572e29-f895-4b1f-b676-2c039bd1a241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72275ed9c4594ca5b22a44cecfb6151d)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:12.988460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14572e29-f895-4b1f-b676-2c039bd1a241 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:13.043024 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-14572e29-f895-4b1f-b676-2c039bd1a241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72275ed9c4594ca5b22a44cecfb6151d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:13.043687 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-14572e29-f895-4b1f-b676-2c039bd1a241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72275ed9c4594ca5b22a44cecfb6151d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:13.052114 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-14572e29-f895-4b1f-b676-2c039bd1a241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72275ed9c4594ca5b22a44cecfb6151d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:13.067309 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-14572e29-f895-4b1f-b676-2c039bd1a241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72275ed9c4594ca5b22a44cecfb6151d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:13.068047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-14572e29-f895-4b1f-b676-2c039bd1a241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72275ed9c4594ca5b22a44cecfb6151d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:13.087615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1217/2448] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:06:12 2022] DELETE /identity/v3/users/72275ed9c4594ca5b22a44cecfb6151d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:13.089333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1839a5-18c8-4331-b79a-8a7da2da40aa None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:13.089850 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1839a5-18c8-4331-b79a-8a7da2da40aa None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:13.089946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1839a5-18c8-4331-b79a-8a7da2da40aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:13.108719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1c1839a5-18c8-4331-b79a-8a7da2da40aa None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:13.110924 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1c1839a5-18c8-4331-b79a-8a7da2da40aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:13.179951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1232/2449] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:13.190211 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14572e29-f895-4b1f-b676-2c039bd1a241 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:13.191669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-14572e29-f895-4b1f-b676-2c039bd1a241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:13.225102 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:13.262315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14572e29-f895-4b1f-b676-2c039bd1a241 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:13.266724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1ffc7-7b8a-41e9-9de2-4f2804b679b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:13.267626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1ffc7-7b8a-41e9-9de2-4f2804b679b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:13.267955 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1ffc7-7b8a-41e9-9de2-4f2804b679b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:13.268291 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1ffc7-7b8a-41e9-9de2-4f2804b679b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:13.279729 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1ffc7-7b8a-41e9-9de2-4f2804b679b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:13.281992 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1ffc7-7b8a-41e9-9de2-4f2804b679b9 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:13.295583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1218/2450] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:13.747274 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1839a5-18c8-4331-b79a-8a7da2da40aa None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:13.748780 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1c1839a5-18c8-4331-b79a-8a7da2da40aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:13.770733 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1839a5-18c8-4331-b79a-8a7da2da40aa None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:13.776979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ac381-af4b-44af-b4df-835dbda0a45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:13.778463 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ac381-af4b-44af-b4df-835dbda0a45e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:13.779096 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ac381-af4b-44af-b4df-835dbda0a45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:13.779791 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ac381-af4b-44af-b4df-835dbda0a45e admin admin] PATH_INFO: `/v3/projects/17adf3d56ff04eef9e5bc04252cc0c43` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:13.782727 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ac381-af4b-44af-b4df-835dbda0a45e admin admin] RBAC: Authorizing `identity:delete_project(project_id=17adf3d56ff04eef9e5bc04252cc0c43)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:13.785301 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ac381-af4b-44af-b4df-835dbda0a45e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:13.800550 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:13.853487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-082ac381-af4b-44af-b4df-835dbda0a45e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17adf3d56ff04eef9e5bc04252cc0c43', 'request_id': 'req-082ac381-af4b-44af-b4df-835dbda0a45e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:13.863629 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-082ac381-af4b-44af-b4df-835dbda0a45e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17adf3d56ff04eef9e5bc04252cc0c43', 'request_id': 'req-082ac381-af4b-44af-b4df-835dbda0a45e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:13.874413 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-082ac381-af4b-44af-b4df-835dbda0a45e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17adf3d56ff04eef9e5bc04252cc0c43', 'request_id': 'req-082ac381-af4b-44af-b4df-835dbda0a45e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:13.886947 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-082ac381-af4b-44af-b4df-835dbda0a45e admin admin] The token cache is being invalidate because project 17adf3d56ff04eef9e5bc04252cc0c43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:06:13.887362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-082ac381-af4b-44af-b4df-835dbda0a45e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:13.888026 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-082ac381-af4b-44af-b4df-835dbda0a45e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:13.894267 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1233/2451] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:06:13 2022] DELETE /identity/v3/projects/17adf3d56ff04eef9e5bc04252cc0c43 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:13.895552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1ffc7-7b8a-41e9-9de2-4f2804b679b9 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:13.906602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1ffc7-7b8a-41e9-9de2-4f2804b679b9 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:13.996454 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3607730-3ef3-424a-a00f-3be30e974608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:13.999309 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3607730-3ef3-424a-a00f-3be30e974608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:14.000798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3607730-3ef3-424a-a00f-3be30e974608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:14.001372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3607730-3ef3-424a-a00f-3be30e974608 admin admin] PATH_INFO: `/v3/users/0b1914ecb4944f1d8d8983ba3dd9a57d` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:14.002826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3607730-3ef3-424a-a00f-3be30e974608 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b1914ecb4944f1d8d8983ba3dd9a57d)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:14.004761 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3607730-3ef3-424a-a00f-3be30e974608 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:14.068083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d3607730-3ef3-424a-a00f-3be30e974608 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b1914ecb4944f1d8d8983ba3dd9a57d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:14.069086 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d3607730-3ef3-424a-a00f-3be30e974608 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b1914ecb4944f1d8d8983ba3dd9a57d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:14.077335 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d3607730-3ef3-424a-a00f-3be30e974608 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b1914ecb4944f1d8d8983ba3dd9a57d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:14.092984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d3607730-3ef3-424a-a00f-3be30e974608 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b1914ecb4944f1d8d8983ba3dd9a57d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:14.094259 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d3607730-3ef3-424a-a00f-3be30e974608 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b1914ecb4944f1d8d8983ba3dd9a57d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:14.125607 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1219/2452] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:06:13 2022] DELETE /identity/v3/users/0b1914ecb4944f1d8d8983ba3dd9a57d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:14.471465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ac381-af4b-44af-b4df-835dbda0a45e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:14.472998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-082ac381-af4b-44af-b4df-835dbda0a45e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:14.514368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ac381-af4b-44af-b4df-835dbda0a45e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:14.629911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6c2d2-fef1-40a4-a29d-82fe91319ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:14.630820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6c2d2-fef1-40a4-a29d-82fe91319ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:14.631101 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6c2d2-fef1-40a4-a29d-82fe91319ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:14.631374 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6c2d2-fef1-40a4-a29d-82fe91319ced admin admin] PATH_INFO: `/v3/projects/445a3cec63f5411baf11ebfd69e2683a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:14.632798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6c2d2-fef1-40a4-a29d-82fe91319ced admin admin] RBAC: Authorizing `identity:delete_project(project_id=445a3cec63f5411baf11ebfd69e2683a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:14.634971 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6c2d2-fef1-40a4-a29d-82fe91319ced admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:14.701771 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-67a6c2d2-fef1-40a4-a29d-82fe91319ced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '445a3cec63f5411baf11ebfd69e2683a', 'request_id': 'req-67a6c2d2-fef1-40a4-a29d-82fe91319ced'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:14.713387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-67a6c2d2-fef1-40a4-a29d-82fe91319ced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '445a3cec63f5411baf11ebfd69e2683a', 'request_id': 'req-67a6c2d2-fef1-40a4-a29d-82fe91319ced'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:14.724678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-67a6c2d2-fef1-40a4-a29d-82fe91319ced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '445a3cec63f5411baf11ebfd69e2683a', 'request_id': 'req-67a6c2d2-fef1-40a4-a29d-82fe91319ced'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:14.738189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-67a6c2d2-fef1-40a4-a29d-82fe91319ced admin admin] The token cache is being invalidate because project 445a3cec63f5411baf11ebfd69e2683a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:06:14.738479 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-67a6c2d2-fef1-40a4-a29d-82fe91319ced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:14.739211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-67a6c2d2-fef1-40a4-a29d-82fe91319ced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:14.742657 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1234/2453] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:06:14 2022] DELETE /identity/v3/projects/445a3cec63f5411baf11ebfd69e2683a => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:14.745905 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234a5c9-5e92-413e-9903-bcf46ee6bd0a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:14.746194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234a5c9-5e92-413e-9903-bcf46ee6bd0a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:14.747267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234a5c9-5e92-413e-9903-bcf46ee6bd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:14.776020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b234a5c9-5e92-413e-9903-bcf46ee6bd0a None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:14.777887 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b234a5c9-5e92-413e-9903-bcf46ee6bd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:14.840321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:14.883810 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1220/2454] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:14.886209 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f7cde-62b6-4d75-9c54-06c063a707dd None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:14.886633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f7cde-62b6-4d75-9c54-06c063a707dd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:14.887006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f7cde-62b6-4d75-9c54-06c063a707dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:14.925337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f43f7cde-62b6-4d75-9c54-06c063a707dd None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:14.926612 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f43f7cde-62b6-4d75-9c54-06c063a707dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:14.943932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234a5c9-5e92-413e-9903-bcf46ee6bd0a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:14.950889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1235/2455] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:14.965158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234a5c9-5e92-413e-9903-bcf46ee6bd0a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:14.972489 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae5bb7-49ab-4b7d-96aa-3b06669a5012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:14.973445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae5bb7-49ab-4b7d-96aa-3b06669a5012 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:14.973936 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae5bb7-49ab-4b7d-96aa-3b06669a5012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:14.974366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae5bb7-49ab-4b7d-96aa-3b06669a5012 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:14.975561 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae5bb7-49ab-4b7d-96aa-3b06669a5012 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:06:14.976023 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae5bb7-49ab-4b7d-96aa-3b06669a5012 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:14.978100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae5bb7-49ab-4b7d-96aa-3b06669a5012 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:14.997338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1221/2456] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:06:14.998126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f7cde-62b6-4d75-9c54-06c063a707dd None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:15.020728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f7cde-62b6-4d75-9c54-06c063a707dd None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:15.027471 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c5b99-3a6c-4207-9b26-0a30f8b5ec2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:15.028350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c5b99-3a6c-4207-9b26-0a30f8b5ec2a admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:15.031874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c5b99-3a6c-4207-9b26-0a30f8b5ec2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:15.032558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c5b99-3a6c-4207-9b26-0a30f8b5ec2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:15.034822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c5b99-3a6c-4207-9b26-0a30f8b5ec2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:15.038471 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c5b99-3a6c-4207-9b26-0a30f8b5ec2a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:15.068731 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:15.087719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1236/2457] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:06:14 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:06:15.089201 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae5bb7-49ab-4b7d-96aa-3b06669a5012 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:15.101061 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae5bb7-49ab-4b7d-96aa-3b06669a5012 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:15.181789 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484caee-7578-455d-a9be-e66d93bbc817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:15.182603 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484caee-7578-455d-a9be-e66d93bbc817 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:15.182955 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484caee-7578-455d-a9be-e66d93bbc817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:15.183483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484caee-7578-455d-a9be-e66d93bbc817 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:15.184252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484caee-7578-455d-a9be-e66d93bbc817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:15.186608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484caee-7578-455d-a9be-e66d93bbc817 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:15.231764 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1222/2458] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:06:15 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:06:15.236464 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c5b99-3a6c-4207-9b26-0a30f8b5ec2a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:15.247913 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c5b99-3a6c-4207-9b26-0a30f8b5ec2a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:15.256515 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b7108-a3c9-4e3d-9821-ba290e8fcb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:15.257795 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b7108-a3c9-4e3d-9821-ba290e8fcb35 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:15.258089 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b7108-a3c9-4e3d-9821-ba290e8fcb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:15.258368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b7108-a3c9-4e3d-9821-ba290e8fcb35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:15.259131 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b7108-a3c9-4e3d-9821-ba290e8fcb35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:15.260803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b7108-a3c9-4e3d-9821-ba290e8fcb35 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:15.275170 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1237/2459] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:06:15.279711 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484caee-7578-455d-a9be-e66d93bbc817 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:15.303076 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484caee-7578-455d-a9be-e66d93bbc817 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:15.310055 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22075bde-a538-428d-9b8e-52ce92fc5810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:15.310255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22075bde-a538-428d-9b8e-52ce92fc5810 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:15.310465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22075bde-a538-428d-9b8e-52ce92fc5810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:15.310729 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22075bde-a538-428d-9b8e-52ce92fc5810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:15.311619 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22075bde-a538-428d-9b8e-52ce92fc5810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:15.314111 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22075bde-a538-428d-9b8e-52ce92fc5810 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:15.327298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1223/2460] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:06:15.331388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b7108-a3c9-4e3d-9821-ba290e8fcb35 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:15.344253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b7108-a3c9-4e3d-9821-ba290e8fcb35 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:15.351946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc566fc-1ef2-40be-be9c-eb7e1dbce768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:15.357779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc566fc-1ef2-40be-be9c-eb7e1dbce768 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:15.357779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc566fc-1ef2-40be-be9c-eb7e1dbce768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:15.357779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc566fc-1ef2-40be-be9c-eb7e1dbce768 admin admin] PATH_INFO: `/v3/projects/3a7ff3a6ba5f4f3da833cbe7ee28a82a/users/dfcfc60434a94088a2630ee447f89e08/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:15.377928 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc566fc-1ef2-40be-be9c-eb7e1dbce768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a7ff3a6ba5f4f3da833cbe7ee28a82a, user_id=dfcfc60434a94088a2630ee447f89e08, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:15.381698 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc566fc-1ef2-40be-be9c-eb7e1dbce768 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:15.399838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1238/2461] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:06:15 2022] PUT /identity/v3/projects/3a7ff3a6ba5f4f3da833cbe7ee28a82a/users/dfcfc60434a94088a2630ee447f89e08/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:06:15.413047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df62f59-c9ac-4d9a-a393-9f0f6063bfd5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:15.413609 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df62f59-c9ac-4d9a-a393-9f0f6063bfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:15.413894 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df62f59-c9ac-4d9a-a393-9f0f6063bfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:15.445780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-0df62f59-c9ac-4d9a-a393-9f0f6063bfd5 None None] MFA Rules not processed for user `dfcfc60434a94088a2630ee447f89e08`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:15.447180 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0df62f59-c9ac-4d9a-a393-9f0f6063bfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:15.529437 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1224/2462] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:06:15.536153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc566fc-1ef2-40be-be9c-eb7e1dbce768 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:15.609718 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc566fc-1ef2-40be-be9c-eb7e1dbce768 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:15.615133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d9b01-074c-4c00-bc95-00a1b75e43a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:15.615781 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d9b01-074c-4c00-bc95-00a1b75e43a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:15.616002 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d9b01-074c-4c00-bc95-00a1b75e43a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:15.616280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d9b01-074c-4c00-bc95-00a1b75e43a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:15.617671 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d9b01-074c-4c00-bc95-00a1b75e43a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:15.619709 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d9b01-074c-4c00-bc95-00a1b75e43a1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:15.665955 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1239/2463] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:06:15 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:06:15.669141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df62f59-c9ac-4d9a-a393-9f0f6063bfd5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:15.684561 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df62f59-c9ac-4d9a-a393-9f0f6063bfd5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:15.744018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:15.776787 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a451e4-4e90-4759-b487-13749e30aff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:15.778084 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a451e4-4e90-4759-b487-13749e30aff2 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:15.778612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a451e4-4e90-4759-b487-13749e30aff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:15.778982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a451e4-4e90-4759-b487-13749e30aff2 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:15.779923 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a451e4-4e90-4759-b487-13749e30aff2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:15.782612 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a451e4-4e90-4759-b487-13749e30aff2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:15.815929 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1225/2464] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:06:15 2022] POST /identity/v3/users => generated 299 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:06:15.819143 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d9b01-074c-4c00-bc95-00a1b75e43a1 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:15.831336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d9b01-074c-4c00-bc95-00a1b75e43a1 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:15.838747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc6fb3-3354-4634-8e90-2a1724352aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:15.839427 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc6fb3-3354-4634-8e90-2a1724352aec admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:15.839854 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc6fb3-3354-4634-8e90-2a1724352aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:15.840152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc6fb3-3354-4634-8e90-2a1724352aec admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:15.840863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc6fb3-3354-4634-8e90-2a1724352aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:15.842555 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc6fb3-3354-4634-8e90-2a1724352aec admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:15.858357 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1240/2465] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:06:15.860802 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a451e4-4e90-4759-b487-13749e30aff2 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:15.872650 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a451e4-4e90-4759-b487-13749e30aff2 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:15.880955 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273efbd-7357-43a1-aaa1-2be407f84f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:15.881997 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273efbd-7357-43a1-aaa1-2be407f84f2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:15.882280 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273efbd-7357-43a1-aaa1-2be407f84f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:15.882546 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273efbd-7357-43a1-aaa1-2be407f84f2b admin admin] PATH_INFO: `/v3/projects/7c428ddef9ce4e3e8564ef0df58afe4a/users/a9953b0a37ee4fd099b47280a2aa0636/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:15.917252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273efbd-7357-43a1-aaa1-2be407f84f2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c428ddef9ce4e3e8564ef0df58afe4a, user_id=a9953b0a37ee4fd099b47280a2aa0636, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:15.920823 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273efbd-7357-43a1-aaa1-2be407f84f2b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:15.947887 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1226/2466] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:06:15 2022] PUT /identity/v3/projects/7c428ddef9ce4e3e8564ef0df58afe4a/users/a9953b0a37ee4fd099b47280a2aa0636/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:15.949693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc6fb3-3354-4634-8e90-2a1724352aec admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:15.963690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc6fb3-3354-4634-8e90-2a1724352aec admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:16.032106 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87831d18-b99b-425a-9baf-ef391c17520f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:16.033420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87831d18-b99b-425a-9baf-ef391c17520f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:16.033420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87831d18-b99b-425a-9baf-ef391c17520f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:16.033420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87831d18-b99b-425a-9baf-ef391c17520f admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:16.034160 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87831d18-b99b-425a-9baf-ef391c17520f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:16.036191 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87831d18-b99b-425a-9baf-ef391c17520f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:16.047381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1241/2467] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:06:16.051361 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273efbd-7357-43a1-aaa1-2be407f84f2b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:16.067845 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273efbd-7357-43a1-aaa1-2be407f84f2b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:16.073888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f34ab6-189a-4045-ab3f-010f5c997a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:16.074963 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f34ab6-189a-4045-ab3f-010f5c997a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:16.075281 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f34ab6-189a-4045-ab3f-010f5c997a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:16.075572 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f34ab6-189a-4045-ab3f-010f5c997a75 admin admin] PATH_INFO: `/v3/projects/7c428ddef9ce4e3e8564ef0df58afe4a/users/a9953b0a37ee4fd099b47280a2aa0636/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:16.077941 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f34ab6-189a-4045-ab3f-010f5c997a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c428ddef9ce4e3e8564ef0df58afe4a, user_id=a9953b0a37ee4fd099b47280a2aa0636, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:16.082035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f34ab6-189a-4045-ab3f-010f5c997a75 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:16.100734 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1227/2468] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:06:16 2022] PUT /identity/v3/projects/7c428ddef9ce4e3e8564ef0df58afe4a/users/a9953b0a37ee4fd099b47280a2aa0636/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:16.113211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7b95a-06e6-4703-8e5d-fa9de2a56628 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:16.113681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7b95a-06e6-4703-8e5d-fa9de2a56628 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:16.113681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7b95a-06e6-4703-8e5d-fa9de2a56628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:16.157995 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-72d7b95a-06e6-4703-8e5d-fa9de2a56628 None None] MFA Rules not processed for user `a9953b0a37ee4fd099b47280a2aa0636`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:16.159327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-72d7b95a-06e6-4703-8e5d-fa9de2a56628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:16.188792 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:16.264649 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1242/2469] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:16.283869 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f34ab6-189a-4045-ab3f-010f5c997a75 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:16.286428 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-53f34ab6-189a-4045-ab3f-010f5c997a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:16.356580 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f34ab6-189a-4045-ab3f-010f5c997a75 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:16.363716 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71acac1c-0bcb-4d50-8c5a-48c2e239ada2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:16.364380 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acac1c-0bcb-4d50-8c5a-48c2e239ada2 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:16.364663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acac1c-0bcb-4d50-8c5a-48c2e239ada2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:16.364940 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acac1c-0bcb-4d50-8c5a-48c2e239ada2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:16.375294 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71acac1c-0bcb-4d50-8c5a-48c2e239ada2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:16.376933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71acac1c-0bcb-4d50-8c5a-48c2e239ada2 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:16.390352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1228/2470] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:06:16.423852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7b95a-06e6-4703-8e5d-fa9de2a56628 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:16.522709 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7b95a-06e6-4703-8e5d-fa9de2a56628 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:16.530729 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae0a31-b389-46d8-b2cd-bff759ec9436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:16.533136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae0a31-b389-46d8-b2cd-bff759ec9436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:16.533620 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae0a31-b389-46d8-b2cd-bff759ec9436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:16.534074 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae0a31-b389-46d8-b2cd-bff759ec9436 admin admin] PATH_INFO: `/v3/users/dfcfc60434a94088a2630ee447f89e08` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:16.535472 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae0a31-b389-46d8-b2cd-bff759ec9436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfcfc60434a94088a2630ee447f89e08)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:16.537445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae0a31-b389-46d8-b2cd-bff759ec9436 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:16.597670 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a4ae0a31-b389-46d8-b2cd-bff759ec9436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfcfc60434a94088a2630ee447f89e08'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:16.599522 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a4ae0a31-b389-46d8-b2cd-bff759ec9436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfcfc60434a94088a2630ee447f89e08'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:16.600719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a4ae0a31-b389-46d8-b2cd-bff759ec9436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfcfc60434a94088a2630ee447f89e08'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:16.612824 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a4ae0a31-b389-46d8-b2cd-bff759ec9436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfcfc60434a94088a2630ee447f89e08'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:16.622733 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a4ae0a31-b389-46d8-b2cd-bff759ec9436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfcfc60434a94088a2630ee447f89e08'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:16.650995 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1243/2471] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:06:16 2022] DELETE /identity/v3/users/dfcfc60434a94088a2630ee447f89e08 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:16.655177 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f4d32-808f-4b91-98b1-85b5fd341218 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:16.655468 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f4d32-808f-4b91-98b1-85b5fd341218 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:16.655758 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f4d32-808f-4b91-98b1-85b5fd341218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:16.678519 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-341f4d32-808f-4b91-98b1-85b5fd341218 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:16.679768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-341f4d32-808f-4b91-98b1-85b5fd341218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:16.762492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1229/2472] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:16.769639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae0a31-b389-46d8-b2cd-bff759ec9436 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:16.771249 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a4ae0a31-b389-46d8-b2cd-bff759ec9436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:16.836785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:16.853708 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae0a31-b389-46d8-b2cd-bff759ec9436 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:16.858415 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd218f4-4985-41f8-98c1-4f1a81f490f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:16.859291 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd218f4-4985-41f8-98c1-4f1a81f490f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:16.859756 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd218f4-4985-41f8-98c1-4f1a81f490f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:16.860116 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd218f4-4985-41f8-98c1-4f1a81f490f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:16.872938 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd218f4-4985-41f8-98c1-4f1a81f490f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:16.874406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd218f4-4985-41f8-98c1-4f1a81f490f8 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:16.886750 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1244/2473] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:17.261124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f4d32-808f-4b91-98b1-85b5fd341218 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:17.262722 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-341f4d32-808f-4b91-98b1-85b5fd341218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:17.293558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f4d32-808f-4b91-98b1-85b5fd341218 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:17.314776 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6777d-d21f-4ced-af45-198441d10f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:17.315660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6777d-d21f-4ced-af45-198441d10f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:17.315938 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6777d-d21f-4ced-af45-198441d10f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:17.316247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6777d-d21f-4ced-af45-198441d10f03 admin admin] PATH_INFO: `/v3/projects/3a7ff3a6ba5f4f3da833cbe7ee28a82a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:17.320511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6777d-d21f-4ced-af45-198441d10f03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a7ff3a6ba5f4f3da833cbe7ee28a82a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:17.321251 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6777d-d21f-4ced-af45-198441d10f03 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:17.389591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:17.409057 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e7d6777d-d21f-4ced-af45-198441d10f03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a7ff3a6ba5f4f3da833cbe7ee28a82a', 'request_id': 'req-e7d6777d-d21f-4ced-af45-198441d10f03'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:17.420018 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e7d6777d-d21f-4ced-af45-198441d10f03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a7ff3a6ba5f4f3da833cbe7ee28a82a', 'request_id': 'req-e7d6777d-d21f-4ced-af45-198441d10f03'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:17.427951 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e7d6777d-d21f-4ced-af45-198441d10f03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a7ff3a6ba5f4f3da833cbe7ee28a82a', 'request_id': 'req-e7d6777d-d21f-4ced-af45-198441d10f03'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:17.442460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e7d6777d-d21f-4ced-af45-198441d10f03 admin admin] The token cache is being invalidate because project 3a7ff3a6ba5f4f3da833cbe7ee28a82a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:06:17.442768 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e7d6777d-d21f-4ced-af45-198441d10f03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:17.443480 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e7d6777d-d21f-4ced-af45-198441d10f03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:17.445519 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1230/2474] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:06:17 2022] DELETE /identity/v3/projects/3a7ff3a6ba5f4f3da833cbe7ee28a82a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:17.449217 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd218f4-4985-41f8-98c1-4f1a81f490f8 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:17.464482 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd218f4-4985-41f8-98c1-4f1a81f490f8 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:17.546258 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c7408-8cfa-43c2-8543-67f6ebef9054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:17.548444 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c7408-8cfa-43c2-8543-67f6ebef9054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:17.548444 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c7408-8cfa-43c2-8543-67f6ebef9054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:17.548444 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c7408-8cfa-43c2-8543-67f6ebef9054 admin admin] PATH_INFO: `/v3/users/a9953b0a37ee4fd099b47280a2aa0636` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:17.549864 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c7408-8cfa-43c2-8543-67f6ebef9054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9953b0a37ee4fd099b47280a2aa0636)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:17.551905 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c7408-8cfa-43c2-8543-67f6ebef9054 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:17.610981 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-589c7408-8cfa-43c2-8543-67f6ebef9054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9953b0a37ee4fd099b47280a2aa0636'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:17.611902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-589c7408-8cfa-43c2-8543-67f6ebef9054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9953b0a37ee4fd099b47280a2aa0636'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:17.612704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-589c7408-8cfa-43c2-8543-67f6ebef9054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9953b0a37ee4fd099b47280a2aa0636'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:17.624405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-589c7408-8cfa-43c2-8543-67f6ebef9054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9953b0a37ee4fd099b47280a2aa0636'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:17.632430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-589c7408-8cfa-43c2-8543-67f6ebef9054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9953b0a37ee4fd099b47280a2aa0636'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:17.652864 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1245/2475] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:06:17 2022] DELETE /identity/v3/users/a9953b0a37ee4fd099b47280a2aa0636 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:17.986269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6777d-d21f-4ced-af45-198441d10f03 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:17.988086 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e7d6777d-d21f-4ced-af45-198441d10f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:18.008858 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6777d-d21f-4ced-af45-198441d10f03 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:18.089834 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58e637-753e-4cfa-94dd-b70ed2af2f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:18.090894 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58e637-753e-4cfa-94dd-b70ed2af2f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:18.091169 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58e637-753e-4cfa-94dd-b70ed2af2f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:18.091450 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58e637-753e-4cfa-94dd-b70ed2af2f8d admin admin] PATH_INFO: `/v3/projects/7c428ddef9ce4e3e8564ef0df58afe4a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:18.092664 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58e637-753e-4cfa-94dd-b70ed2af2f8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c428ddef9ce4e3e8564ef0df58afe4a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:18.095600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58e637-753e-4cfa-94dd-b70ed2af2f8d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:18.150195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7b58e637-753e-4cfa-94dd-b70ed2af2f8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c428ddef9ce4e3e8564ef0df58afe4a', 'request_id': 'req-7b58e637-753e-4cfa-94dd-b70ed2af2f8d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:18.159793 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7b58e637-753e-4cfa-94dd-b70ed2af2f8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c428ddef9ce4e3e8564ef0df58afe4a', 'request_id': 'req-7b58e637-753e-4cfa-94dd-b70ed2af2f8d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:18.169696 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7b58e637-753e-4cfa-94dd-b70ed2af2f8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c428ddef9ce4e3e8564ef0df58afe4a', 'request_id': 'req-7b58e637-753e-4cfa-94dd-b70ed2af2f8d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:18.182811 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7b58e637-753e-4cfa-94dd-b70ed2af2f8d admin admin] The token cache is being invalidate because project 7c428ddef9ce4e3e8564ef0df58afe4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:06:18.183285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7b58e637-753e-4cfa-94dd-b70ed2af2f8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:18.184002 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7b58e637-753e-4cfa-94dd-b70ed2af2f8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:18.185791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1231/2476] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:06:17 2022] DELETE /identity/v3/projects/7c428ddef9ce4e3e8564ef0df58afe4a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:18.191956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2976b-1388-442e-ae8e-48b4fc5e7441 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:18.192250 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2976b-1388-442e-ae8e-48b4fc5e7441 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:18.192527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2976b-1388-442e-ae8e-48b4fc5e7441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:18.212409 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e9f2976b-1388-442e-ae8e-48b4fc5e7441 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:18.213758 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e9f2976b-1388-442e-ae8e-48b4fc5e7441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:18.286624 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1246/2477] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:18.292864 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa906c-3791-4be5-ab01-2af041f8f6bd None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:18.293441 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa906c-3791-4be5-ab01-2af041f8f6bd None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:18.293603 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa906c-3791-4be5-ab01-2af041f8f6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:18.315311 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-75aa906c-3791-4be5-ab01-2af041f8f6bd None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:18.316646 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-75aa906c-3791-4be5-ab01-2af041f8f6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:18.327832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1232/2478] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:18.329131 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2976b-1388-442e-ae8e-48b4fc5e7441 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:18.334007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:18.339766 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2976b-1388-442e-ae8e-48b4fc5e7441 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:18.348222 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9267c467-e340-408f-af75-c5cdfe4189c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:18.349594 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267c467-e340-408f-af75-c5cdfe4189c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:18.349930 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267c467-e340-408f-af75-c5cdfe4189c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:18.350206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267c467-e340-408f-af75-c5cdfe4189c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:18.350990 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9267c467-e340-408f-af75-c5cdfe4189c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:06:18.351295 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9267c467-e340-408f-af75-c5cdfe4189c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:18.355302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9267c467-e340-408f-af75-c5cdfe4189c8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:18.371867 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1247/2479] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:06:18.379677 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa906c-3791-4be5-ab01-2af041f8f6bd None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:18.390333 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa906c-3791-4be5-ab01-2af041f8f6bd None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:18.395990 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fb28d-8e7e-423e-9796-163d4869824c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:18.397564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fb28d-8e7e-423e-9796-163d4869824c admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:18.397958 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fb28d-8e7e-423e-9796-163d4869824c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:18.398360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fb28d-8e7e-423e-9796-163d4869824c admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:18.399194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966fb28d-8e7e-423e-9796-163d4869824c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:18.401423 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966fb28d-8e7e-423e-9796-163d4869824c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:18.437692 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1233/2480] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:06:18 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:06:18.445070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9267c467-e340-408f-af75-c5cdfe4189c8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:18.462811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9267c467-e340-408f-af75-c5cdfe4189c8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:18.546927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3f85d-8b95-40f8-ae47-3053f9a1fd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:18.548010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3f85d-8b95-40f8-ae47-3053f9a1fd13 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:18.548010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3f85d-8b95-40f8-ae47-3053f9a1fd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:18.548181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3f85d-8b95-40f8-ae47-3053f9a1fd13 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:18.548929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3f85d-8b95-40f8-ae47-3053f9a1fd13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:18.551238 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3f85d-8b95-40f8-ae47-3053f9a1fd13 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:18.576778 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1248/2481] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:06:18 2022] POST /identity/v3/users => generated 294 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:06:18.583058 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fb28d-8e7e-423e-9796-163d4869824c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:18.607756 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fb28d-8e7e-423e-9796-163d4869824c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:18.617404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a618a2-4ad8-4fb7-a475-fa4fd048efe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:18.618903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a618a2-4ad8-4fb7-a475-fa4fd048efe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:18.619195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a618a2-4ad8-4fb7-a475-fa4fd048efe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:18.619472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a618a2-4ad8-4fb7-a475-fa4fd048efe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:18.620282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a618a2-4ad8-4fb7-a475-fa4fd048efe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:18.623489 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a618a2-4ad8-4fb7-a475-fa4fd048efe8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:18.642680 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1234/2482] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:06:18.646518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3f85d-8b95-40f8-ae47-3053f9a1fd13 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:18.668675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3f85d-8b95-40f8-ae47-3053f9a1fd13 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:18.674053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7e4c6-b18b-487e-ba8c-9cdea22e6717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:18.674949 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7e4c6-b18b-487e-ba8c-9cdea22e6717 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:18.675075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7e4c6-b18b-487e-ba8c-9cdea22e6717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:18.675319 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7e4c6-b18b-487e-ba8c-9cdea22e6717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:18.676200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7e4c6-b18b-487e-ba8c-9cdea22e6717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:18.678046 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7e4c6-b18b-487e-ba8c-9cdea22e6717 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:18.694265 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1249/2483] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:06:18.696266 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a618a2-4ad8-4fb7-a475-fa4fd048efe8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:18.708370 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a618a2-4ad8-4fb7-a475-fa4fd048efe8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:18.713472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e659f7-9b8e-44ff-95c2-1d40869a3ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:18.714454 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e659f7-9b8e-44ff-95c2-1d40869a3ad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:18.714842 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e659f7-9b8e-44ff-95c2-1d40869a3ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:18.715197 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e659f7-9b8e-44ff-95c2-1d40869a3ad6 admin admin] PATH_INFO: `/v3/projects/b44cd7b32be742e2b6f6a8498af62bad/users/b18b397cf2df47d3ba854c8749d66942/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:18.724702 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:18.733603 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e659f7-9b8e-44ff-95c2-1d40869a3ad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44cd7b32be742e2b6f6a8498af62bad, user_id=b18b397cf2df47d3ba854c8749d66942, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:18.742767 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e659f7-9b8e-44ff-95c2-1d40869a3ad6 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:18.773934 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1235/2484] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:06:18 2022] PUT /identity/v3/projects/b44cd7b32be742e2b6f6a8498af62bad/users/b18b397cf2df47d3ba854c8749d66942/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:06:18.776209 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988ad39-5868-4ce9-a803-5ad37218b717 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:18.776557 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988ad39-5868-4ce9-a803-5ad37218b717 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:18.776904 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988ad39-5868-4ce9-a803-5ad37218b717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:18.808338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1988ad39-5868-4ce9-a803-5ad37218b717 None None] MFA Rules not processed for user `b18b397cf2df47d3ba854c8749d66942`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:18.810410 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1988ad39-5868-4ce9-a803-5ad37218b717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:18.888262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1250/2485] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:18.893965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e659f7-9b8e-44ff-95c2-1d40869a3ad6 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:18.966823 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e659f7-9b8e-44ff-95c2-1d40869a3ad6 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:18.972764 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ae234-36a8-4f7e-b1fa-9b9eeb5622e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:18.973517 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ae234-36a8-4f7e-b1fa-9b9eeb5622e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:18.973874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ae234-36a8-4f7e-b1fa-9b9eeb5622e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:18.974223 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ae234-36a8-4f7e-b1fa-9b9eeb5622e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:18.975056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ae234-36a8-4f7e-b1fa-9b9eeb5622e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:18.977128 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ae234-36a8-4f7e-b1fa-9b9eeb5622e8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:19.005266 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1236/2486] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:06:18 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:06:19.009287 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988ad39-5868-4ce9-a803-5ad37218b717 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:19.022093 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988ad39-5868-4ce9-a803-5ad37218b717 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:19.082503 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:19.090538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256375d2-b119-49f7-b391-881d8c810d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:19.091449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256375d2-b119-49f7-b391-881d8c810d98 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:19.091743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256375d2-b119-49f7-b391-881d8c810d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:19.092179 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256375d2-b119-49f7-b391-881d8c810d98 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:19.093113 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256375d2-b119-49f7-b391-881d8c810d98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:19.096383 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256375d2-b119-49f7-b391-881d8c810d98 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:19.127625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1251/2487] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:06:19 2022] POST /identity/v3/users => generated 299 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:06:19.128582 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ae234-36a8-4f7e-b1fa-9b9eeb5622e8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:19.139668 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ae234-36a8-4f7e-b1fa-9b9eeb5622e8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:19.146362 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e136a4-b850-4bb4-9dbc-82cb40f88564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:19.147087 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e136a4-b850-4bb4-9dbc-82cb40f88564 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:19.147371 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e136a4-b850-4bb4-9dbc-82cb40f88564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:19.147654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e136a4-b850-4bb4-9dbc-82cb40f88564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:19.148390 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e136a4-b850-4bb4-9dbc-82cb40f88564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:19.150249 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e136a4-b850-4bb4-9dbc-82cb40f88564 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:19.162642 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1237/2488] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:06:19.166156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256375d2-b119-49f7-b391-881d8c810d98 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:19.176538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256375d2-b119-49f7-b391-881d8c810d98 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:19.181241 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df860d-26e1-4042-9386-db9a1df8a92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:19.182150 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df860d-26e1-4042-9386-db9a1df8a92c admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:19.182432 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df860d-26e1-4042-9386-db9a1df8a92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:19.182751 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df860d-26e1-4042-9386-db9a1df8a92c admin admin] PATH_INFO: `/v3/projects/d554d25a775e40ca81b0d26ce4a6a8e7/users/2ba9b0ec581f4dc7a50683bc26d4a749/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:19.199419 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df860d-26e1-4042-9386-db9a1df8a92c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d554d25a775e40ca81b0d26ce4a6a8e7, user_id=2ba9b0ec581f4dc7a50683bc26d4a749, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:19.203635 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df860d-26e1-4042-9386-db9a1df8a92c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:19.219094 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1252/2489] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:06:19 2022] PUT /identity/v3/projects/d554d25a775e40ca81b0d26ce4a6a8e7/users/2ba9b0ec581f4dc7a50683bc26d4a749/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:19.223657 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e136a4-b850-4bb4-9dbc-82cb40f88564 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:19.235719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e136a4-b850-4bb4-9dbc-82cb40f88564 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:19.295195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f6744-a1c0-42d2-8f95-45b56475f216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:19.295899 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f6744-a1c0-42d2-8f95-45b56475f216 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:19.296125 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f6744-a1c0-42d2-8f95-45b56475f216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:19.296429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f6744-a1c0-42d2-8f95-45b56475f216 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:19.297134 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f6744-a1c0-42d2-8f95-45b56475f216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:19.298752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f6744-a1c0-42d2-8f95-45b56475f216 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:19.308754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1238/2490] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:06:19.313906 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df860d-26e1-4042-9386-db9a1df8a92c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:19.324426 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df860d-26e1-4042-9386-db9a1df8a92c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:19.329729 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda088b-efc8-4fad-9fec-6507be7b4e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:19.330749 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda088b-efc8-4fad-9fec-6507be7b4e66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:19.331056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda088b-efc8-4fad-9fec-6507be7b4e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:19.331336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda088b-efc8-4fad-9fec-6507be7b4e66 admin admin] PATH_INFO: `/v3/projects/d554d25a775e40ca81b0d26ce4a6a8e7/users/2ba9b0ec581f4dc7a50683bc26d4a749/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:19.333304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda088b-efc8-4fad-9fec-6507be7b4e66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d554d25a775e40ca81b0d26ce4a6a8e7, user_id=2ba9b0ec581f4dc7a50683bc26d4a749, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:19.335983 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda088b-efc8-4fad-9fec-6507be7b4e66 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:19.346379 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1253/2491] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:06:19 2022] PUT /identity/v3/projects/d554d25a775e40ca81b0d26ce4a6a8e7/users/2ba9b0ec581f4dc7a50683bc26d4a749/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:19.356548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e6637-61a4-4d42-a6ff-c7ca7cfcf3a1 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:19.356839 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e6637-61a4-4d42-a6ff-c7ca7cfcf3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:19.357119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e6637-61a4-4d42-a6ff-c7ca7cfcf3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:19.384407 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-202e6637-61a4-4d42-a6ff-c7ca7cfcf3a1 None None] MFA Rules not processed for user `2ba9b0ec581f4dc7a50683bc26d4a749`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:19.385608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-202e6637-61a4-4d42-a6ff-c7ca7cfcf3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:19.431666 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:19.456694 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1239/2492] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:19.467760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda088b-efc8-4fad-9fec-6507be7b4e66 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:19.469472 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4eda088b-efc8-4fad-9fec-6507be7b4e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:19.527990 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda088b-efc8-4fad-9fec-6507be7b4e66 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:19.533251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83262ac2-f099-432f-8188-1f3a6c6e373b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:19.534302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83262ac2-f099-432f-8188-1f3a6c6e373b service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:19.534802 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83262ac2-f099-432f-8188-1f3a6c6e373b service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:19.535249 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83262ac2-f099-432f-8188-1f3a6c6e373b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:19.545467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83262ac2-f099-432f-8188-1f3a6c6e373b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:19.547509 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83262ac2-f099-432f-8188-1f3a6c6e373b service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:19.557025 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1254/2493] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:19.622123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e6637-61a4-4d42-a6ff-c7ca7cfcf3a1 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:19.691015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e6637-61a4-4d42-a6ff-c7ca7cfcf3a1 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:19.695979 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076bff7b-f3be-4562-b2b5-1700c2a58c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:19.696864 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bff7b-f3be-4562-b2b5-1700c2a58c77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:19.697150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bff7b-f3be-4562-b2b5-1700c2a58c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:19.697426 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bff7b-f3be-4562-b2b5-1700c2a58c77 admin admin] PATH_INFO: `/v3/users/b18b397cf2df47d3ba854c8749d66942` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:19.698682 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076bff7b-f3be-4562-b2b5-1700c2a58c77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b18b397cf2df47d3ba854c8749d66942)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:19.700602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076bff7b-f3be-4562-b2b5-1700c2a58c77 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:19.754360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-076bff7b-f3be-4562-b2b5-1700c2a58c77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b18b397cf2df47d3ba854c8749d66942'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:19.755026 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-076bff7b-f3be-4562-b2b5-1700c2a58c77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b18b397cf2df47d3ba854c8749d66942'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:19.762019 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-076bff7b-f3be-4562-b2b5-1700c2a58c77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b18b397cf2df47d3ba854c8749d66942'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:19.774025 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-076bff7b-f3be-4562-b2b5-1700c2a58c77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b18b397cf2df47d3ba854c8749d66942'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:19.775003 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-076bff7b-f3be-4562-b2b5-1700c2a58c77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b18b397cf2df47d3ba854c8749d66942'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:19.788935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1240/2494] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:06:19 2022] DELETE /identity/v3/users/b18b397cf2df47d3ba854c8749d66942 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:19.793621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80e9eb-3f21-44ec-93f0-e3e957fe92cc None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:19.793915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80e9eb-3f21-44ec-93f0-e3e957fe92cc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:19.794193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80e9eb-3f21-44ec-93f0-e3e957fe92cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:19.814527 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-de80e9eb-3f21-44ec-93f0-e3e957fe92cc None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:19.815826 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-de80e9eb-3f21-44ec-93f0-e3e957fe92cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:19.888419 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1255/2495] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:19.898264 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076bff7b-f3be-4562-b2b5-1700c2a58c77 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:19.899977 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-076bff7b-f3be-4562-b2b5-1700c2a58c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:19.971543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076bff7b-f3be-4562-b2b5-1700c2a58c77 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:19.975286 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0e6e6-2d02-41b0-87a3-8e1f997813a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:19.975928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0e6e6-2d02-41b0-87a3-8e1f997813a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:19.976212 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0e6e6-2d02-41b0-87a3-8e1f997813a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:19.976486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0e6e6-2d02-41b0-87a3-8e1f997813a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:19.986948 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0e6e6-2d02-41b0-87a3-8e1f997813a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:19.988381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0e6e6-2d02-41b0-87a3-8e1f997813a8 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:19.997959 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1241/2496] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:20.445083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80e9eb-3f21-44ec-93f0-e3e957fe92cc None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:20.446548 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-de80e9eb-3f21-44ec-93f0-e3e957fe92cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:20.460061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80e9eb-3f21-44ec-93f0-e3e957fe92cc None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:20.464783 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f5f99-71fb-48ec-b450-10ded63771dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:20.465642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f5f99-71fb-48ec-b450-10ded63771dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:20.465967 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f5f99-71fb-48ec-b450-10ded63771dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:20.466246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f5f99-71fb-48ec-b450-10ded63771dd admin admin] PATH_INFO: `/v3/projects/b44cd7b32be742e2b6f6a8498af62bad` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:20.467604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f5f99-71fb-48ec-b450-10ded63771dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b44cd7b32be742e2b6f6a8498af62bad)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:20.470308 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f5f99-71fb-48ec-b450-10ded63771dd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:20.517252 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:20.518469 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-553f5f99-71fb-48ec-b450-10ded63771dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b44cd7b32be742e2b6f6a8498af62bad', 'request_id': 'req-553f5f99-71fb-48ec-b450-10ded63771dd'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:20.526045 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-553f5f99-71fb-48ec-b450-10ded63771dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b44cd7b32be742e2b6f6a8498af62bad', 'request_id': 'req-553f5f99-71fb-48ec-b450-10ded63771dd'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:20.534092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-553f5f99-71fb-48ec-b450-10ded63771dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b44cd7b32be742e2b6f6a8498af62bad', 'request_id': 'req-553f5f99-71fb-48ec-b450-10ded63771dd'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:20.544827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-553f5f99-71fb-48ec-b450-10ded63771dd admin admin] The token cache is being invalidate because project b44cd7b32be742e2b6f6a8498af62bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:06:20.545067 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-553f5f99-71fb-48ec-b450-10ded63771dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:20.545698 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-553f5f99-71fb-48ec-b450-10ded63771dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:20.547789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1256/2497] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:06:20 2022] DELETE /identity/v3/projects/b44cd7b32be742e2b6f6a8498af62bad => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:20.551863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0e6e6-2d02-41b0-87a3-8e1f997813a8 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:20.557915 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:20.562902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0e6e6-2d02-41b0-87a3-8e1f997813a8 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:20.626564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c795c63-75f4-4632-96aa-27d90ddd88f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:20.627632 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c795c63-75f4-4632-96aa-27d90ddd88f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:20.627921 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c795c63-75f4-4632-96aa-27d90ddd88f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:20.628202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c795c63-75f4-4632-96aa-27d90ddd88f5 admin admin] PATH_INFO: `/v3/users/2ba9b0ec581f4dc7a50683bc26d4a749` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:20.629407 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c795c63-75f4-4632-96aa-27d90ddd88f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ba9b0ec581f4dc7a50683bc26d4a749)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:20.631438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c795c63-75f4-4632-96aa-27d90ddd88f5 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:20.680389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8c795c63-75f4-4632-96aa-27d90ddd88f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ba9b0ec581f4dc7a50683bc26d4a749'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:20.681041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8c795c63-75f4-4632-96aa-27d90ddd88f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ba9b0ec581f4dc7a50683bc26d4a749'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:20.688688 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8c795c63-75f4-4632-96aa-27d90ddd88f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ba9b0ec581f4dc7a50683bc26d4a749'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:20.700261 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8c795c63-75f4-4632-96aa-27d90ddd88f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ba9b0ec581f4dc7a50683bc26d4a749'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:20.700920 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8c795c63-75f4-4632-96aa-27d90ddd88f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ba9b0ec581f4dc7a50683bc26d4a749'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:20.714803 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1242/2498] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:06:20 2022] DELETE /identity/v3/users/2ba9b0ec581f4dc7a50683bc26d4a749 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:21.126102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f5f99-71fb-48ec-b450-10ded63771dd admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:21.128726 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-553f5f99-71fb-48ec-b450-10ded63771dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:21.148693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f5f99-71fb-48ec-b450-10ded63771dd admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:21.232948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a12ca-dad5-4af5-8451-bd19ce105d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:21.234365 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a12ca-dad5-4af5-8451-bd19ce105d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:21.234749 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a12ca-dad5-4af5-8451-bd19ce105d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:21.235379 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a12ca-dad5-4af5-8451-bd19ce105d16 admin admin] PATH_INFO: `/v3/projects/d554d25a775e40ca81b0d26ce4a6a8e7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:21.236478 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a12ca-dad5-4af5-8451-bd19ce105d16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d554d25a775e40ca81b0d26ce4a6a8e7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:21.238373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a12ca-dad5-4af5-8451-bd19ce105d16 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:21.293912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-546a12ca-dad5-4af5-8451-bd19ce105d16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd554d25a775e40ca81b0d26ce4a6a8e7', 'request_id': 'req-546a12ca-dad5-4af5-8451-bd19ce105d16'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:21.302490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-546a12ca-dad5-4af5-8451-bd19ce105d16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd554d25a775e40ca81b0d26ce4a6a8e7', 'request_id': 'req-546a12ca-dad5-4af5-8451-bd19ce105d16'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:21.309521 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-546a12ca-dad5-4af5-8451-bd19ce105d16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd554d25a775e40ca81b0d26ce4a6a8e7', 'request_id': 'req-546a12ca-dad5-4af5-8451-bd19ce105d16'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:21.321114 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-546a12ca-dad5-4af5-8451-bd19ce105d16 admin admin] The token cache is being invalidate because project d554d25a775e40ca81b0d26ce4a6a8e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:06:21.321363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-546a12ca-dad5-4af5-8451-bd19ce105d16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:21.322070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-546a12ca-dad5-4af5-8451-bd19ce105d16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:21.323939 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1257/2499] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:06:21 2022] DELETE /identity/v3/projects/d554d25a775e40ca81b0d26ce4a6a8e7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:21.329780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226d31f-f2fd-4049-8633-8f6023f81877 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:21.330070 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226d31f-f2fd-4049-8633-8f6023f81877 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:21.330348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226d31f-f2fd-4049-8633-8f6023f81877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:21.348361 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1226d31f-f2fd-4049-8633-8f6023f81877 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:21.349557 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1226d31f-f2fd-4049-8633-8f6023f81877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:21.410565 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1243/2500] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:06:21.418169 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1ea1a-a32b-48b9-8296-a252f646cb47 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:21.418457 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1ea1a-a32b-48b9-8296-a252f646cb47 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:21.418785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1ea1a-a32b-48b9-8296-a252f646cb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:21.437454 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-85c1ea1a-a32b-48b9-8296-a252f646cb47 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:21.438860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-85c1ea1a-a32b-48b9-8296-a252f646cb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:21.446596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1258/2501] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:21.450695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226d31f-f2fd-4049-8633-8f6023f81877 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:21.461523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226d31f-f2fd-4049-8633-8f6023f81877 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:21.466202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546e67d-c473-4939-ab9b-7205bc155df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:21.466897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546e67d-c473-4939-ab9b-7205bc155df6 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:21.467184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546e67d-c473-4939-ab9b-7205bc155df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:21.467458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546e67d-c473-4939-ab9b-7205bc155df6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:21.468266 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1546e67d-c473-4939-ab9b-7205bc155df6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:06:21.468728 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1546e67d-c473-4939-ab9b-7205bc155df6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:21.470329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1546e67d-c473-4939-ab9b-7205bc155df6 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:21.482105 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1244/2502] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:06:21.486516 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1ea1a-a32b-48b9-8296-a252f646cb47 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:21.496037 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1ea1a-a32b-48b9-8296-a252f646cb47 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:21.501344 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902809a-b27b-4904-82b9-4d5f4607cf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:21.501993 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902809a-b27b-4904-82b9-4d5f4607cf78 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:21.502306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902809a-b27b-4904-82b9-4d5f4607cf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:21.502602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902809a-b27b-4904-82b9-4d5f4607cf78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:21.503312 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2902809a-b27b-4904-82b9-4d5f4607cf78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:21.505325 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2902809a-b27b-4904-82b9-4d5f4607cf78 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:21.532277 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1259/2503] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:06:21 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:06:21.536362 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546e67d-c473-4939-ab9b-7205bc155df6 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:21.546143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546e67d-c473-4939-ab9b-7205bc155df6 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:21.554916 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:21.610600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf00e93-eb25-4406-9a91-bc1299e27011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:21.611876 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf00e93-eb25-4406-9a91-bc1299e27011 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:21.612801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf00e93-eb25-4406-9a91-bc1299e27011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:21.613086 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf00e93-eb25-4406-9a91-bc1299e27011 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:21.613812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf00e93-eb25-4406-9a91-bc1299e27011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:21.615760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf00e93-eb25-4406-9a91-bc1299e27011 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:21.636263 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1245/2504] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:06:21 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:06:21.641178 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902809a-b27b-4904-82b9-4d5f4607cf78 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:21.651558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902809a-b27b-4904-82b9-4d5f4607cf78 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:21.656198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861499b4-72fb-410a-bc5e-2b1a5666204d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:21.656849 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861499b4-72fb-410a-bc5e-2b1a5666204d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:21.657136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861499b4-72fb-410a-bc5e-2b1a5666204d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:21.657415 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861499b4-72fb-410a-bc5e-2b1a5666204d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:21.658145 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861499b4-72fb-410a-bc5e-2b1a5666204d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:21.659777 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861499b4-72fb-410a-bc5e-2b1a5666204d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:21.670180 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1260/2505] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:06:21.674156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf00e93-eb25-4406-9a91-bc1299e27011 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:21.684183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf00e93-eb25-4406-9a91-bc1299e27011 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:21.689559 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b9729-a85b-45e0-af62-ccf4c44c18d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:21.690205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b9729-a85b-45e0-af62-ccf4c44c18d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:21.690486 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b9729-a85b-45e0-af62-ccf4c44c18d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:21.690789 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b9729-a85b-45e0-af62-ccf4c44c18d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:21.691503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b9729-a85b-45e0-af62-ccf4c44c18d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:21.693088 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b9729-a85b-45e0-af62-ccf4c44c18d2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:21.703792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1246/2506] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:06:21.708041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861499b4-72fb-410a-bc5e-2b1a5666204d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:21.717938 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861499b4-72fb-410a-bc5e-2b1a5666204d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:21.722996 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaded02-51d5-4e11-aaba-642854bc31fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:21.723868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaded02-51d5-4e11-aaba-642854bc31fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:21.724150 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaded02-51d5-4e11-aaba-642854bc31fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:21.724424 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaded02-51d5-4e11-aaba-642854bc31fd admin admin] PATH_INFO: `/v3/projects/423afd79d5564aea82943b0cd93dd128/users/15e25d4ef32e4deab6401f23e31a3e0b/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:21.739450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaded02-51d5-4e11-aaba-642854bc31fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=423afd79d5564aea82943b0cd93dd128, user_id=15e25d4ef32e4deab6401f23e31a3e0b, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:21.742133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaded02-51d5-4e11-aaba-642854bc31fd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:21.755539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1261/2507] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:06:21 2022] PUT /identity/v3/projects/423afd79d5564aea82943b0cd93dd128/users/15e25d4ef32e4deab6401f23e31a3e0b/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:21.766860 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1114a53-84fc-41b7-be16-21e7aafa184f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:21.767181 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1114a53-84fc-41b7-be16-21e7aafa184f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:21.767465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1114a53-84fc-41b7-be16-21e7aafa184f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:21.795091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e1114a53-84fc-41b7-be16-21e7aafa184f None None] MFA Rules not processed for user `15e25d4ef32e4deab6401f23e31a3e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:21.796836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e1114a53-84fc-41b7-be16-21e7aafa184f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:21.862613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1247/2508] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:21.867102 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaded02-51d5-4e11-aaba-642854bc31fd admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:21.871510 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:21.932970 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaded02-51d5-4e11-aaba-642854bc31fd admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:21.938309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651ad69-b4a6-48c9-be56-17e0be859aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:21.939047 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651ad69-b4a6-48c9-be56-17e0be859aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:21.939338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651ad69-b4a6-48c9-be56-17e0be859aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:21.939619 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651ad69-b4a6-48c9-be56-17e0be859aa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:21.940318 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651ad69-b4a6-48c9-be56-17e0be859aa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:21.942227 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651ad69-b4a6-48c9-be56-17e0be859aa0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:21.968935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1262/2509] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:06:21 2022] POST /identity/v3/projects => generated 389 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:06:21.973119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1114a53-84fc-41b7-be16-21e7aafa184f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:21.984913 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1114a53-84fc-41b7-be16-21e7aafa184f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:22.040400 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ccaefe-f36d-475d-8825-73e637e02413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:22.041059 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccaefe-f36d-475d-8825-73e637e02413 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:22.041344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccaefe-f36d-475d-8825-73e637e02413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:22.041619 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccaefe-f36d-475d-8825-73e637e02413 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:22.042299 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ccaefe-f36d-475d-8825-73e637e02413 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:22.044237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ccaefe-f36d-475d-8825-73e637e02413 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:22.065001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1248/2510] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:06:21 2022] POST /identity/v3/users => generated 301 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:06:22.068966 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651ad69-b4a6-48c9-be56-17e0be859aa0 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:22.079752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651ad69-b4a6-48c9-be56-17e0be859aa0 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:22.084776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45282e36-5bc8-4dd7-9c45-dd83da793c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:22.085434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45282e36-5bc8-4dd7-9c45-dd83da793c12 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:22.085711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45282e36-5bc8-4dd7-9c45-dd83da793c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:22.085982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45282e36-5bc8-4dd7-9c45-dd83da793c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:22.086747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45282e36-5bc8-4dd7-9c45-dd83da793c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:22.088332 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45282e36-5bc8-4dd7-9c45-dd83da793c12 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:22.099747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1263/2511] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:06:22.103787 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ccaefe-f36d-475d-8825-73e637e02413 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:22.114646 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ccaefe-f36d-475d-8825-73e637e02413 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:22.119166 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04224ab8-8b67-4a93-9149-89f4bc7ec924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:22.120032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04224ab8-8b67-4a93-9149-89f4bc7ec924 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:22.120310 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04224ab8-8b67-4a93-9149-89f4bc7ec924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:22.120581 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04224ab8-8b67-4a93-9149-89f4bc7ec924 admin admin] PATH_INFO: `/v3/projects/1e556449cbfb46bf92c5fe52a914d03a/users/e0b7cad2f83e4374862208463e21268d/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:22.123976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:22.139379 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04224ab8-8b67-4a93-9149-89f4bc7ec924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e556449cbfb46bf92c5fe52a914d03a, user_id=e0b7cad2f83e4374862208463e21268d, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:22.142080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04224ab8-8b67-4a93-9149-89f4bc7ec924 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:22.155524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1249/2512] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:06:22 2022] PUT /identity/v3/projects/1e556449cbfb46bf92c5fe52a914d03a/users/e0b7cad2f83e4374862208463e21268d/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:22.160542 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45282e36-5bc8-4dd7-9c45-dd83da793c12 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:22.170702 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45282e36-5bc8-4dd7-9c45-dd83da793c12 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:22.229186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63369f-32d9-42ae-bc7a-ba8d3fd1c354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:22.229882 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63369f-32d9-42ae-bc7a-ba8d3fd1c354 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:22.230189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63369f-32d9-42ae-bc7a-ba8d3fd1c354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:22.230423 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63369f-32d9-42ae-bc7a-ba8d3fd1c354 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:22.231181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63369f-32d9-42ae-bc7a-ba8d3fd1c354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:22.232779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63369f-32d9-42ae-bc7a-ba8d3fd1c354 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:22.243860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1264/2513] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:06:22.248699 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04224ab8-8b67-4a93-9149-89f4bc7ec924 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:22.260103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04224ab8-8b67-4a93-9149-89f4bc7ec924 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:22.265066 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0101a31-a028-4877-be3c-2d2eb1128848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:22.266027 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0101a31-a028-4877-be3c-2d2eb1128848 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:22.266378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0101a31-a028-4877-be3c-2d2eb1128848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:22.266759 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0101a31-a028-4877-be3c-2d2eb1128848 admin admin] PATH_INFO: `/v3/projects/1e556449cbfb46bf92c5fe52a914d03a/users/e0b7cad2f83e4374862208463e21268d/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:22.268973 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0101a31-a028-4877-be3c-2d2eb1128848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e556449cbfb46bf92c5fe52a914d03a, user_id=e0b7cad2f83e4374862208463e21268d, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:22.271789 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0101a31-a028-4877-be3c-2d2eb1128848 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:22.284928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1250/2514] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:06:22 2022] PUT /identity/v3/projects/1e556449cbfb46bf92c5fe52a914d03a/users/e0b7cad2f83e4374862208463e21268d/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:22.295294 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441668e-c98f-44be-9527-a02c619e45b5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:22.295593 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441668e-c98f-44be-9527-a02c619e45b5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:22.296029 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441668e-c98f-44be-9527-a02c619e45b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:22.324918 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a441668e-c98f-44be-9527-a02c619e45b5 None None] MFA Rules not processed for user `e0b7cad2f83e4374862208463e21268d`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:22.326284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a441668e-c98f-44be-9527-a02c619e45b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:22.400080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:22.407995 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1265/2515] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:06:22.418334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0101a31-a028-4877-be3c-2d2eb1128848 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:22.420427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e0101a31-a028-4877-be3c-2d2eb1128848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:22.481650 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0101a31-a028-4877-be3c-2d2eb1128848 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:22.487290 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c0a68-cb78-4a1c-83cd-bb7a91a2faf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:22.487954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c0a68-cb78-4a1c-83cd-bb7a91a2faf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:22.488241 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c0a68-cb78-4a1c-83cd-bb7a91a2faf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:22.488522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c0a68-cb78-4a1c-83cd-bb7a91a2faf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:22.501765 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c0a68-cb78-4a1c-83cd-bb7a91a2faf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:22.503326 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c0a68-cb78-4a1c-83cd-bb7a91a2faf6 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:22.514788 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1251/2516] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:22.663847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441668e-c98f-44be-9527-a02c619e45b5 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:22.739952 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441668e-c98f-44be-9527-a02c619e45b5 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:22.744776 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef6317-04a9-4b64-bf9a-42e42e123dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:22.745746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef6317-04a9-4b64-bf9a-42e42e123dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:22.746054 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef6317-04a9-4b64-bf9a-42e42e123dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:22.746381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef6317-04a9-4b64-bf9a-42e42e123dfb admin admin] PATH_INFO: `/v3/users/15e25d4ef32e4deab6401f23e31a3e0b` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:22.747665 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef6317-04a9-4b64-bf9a-42e42e123dfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=15e25d4ef32e4deab6401f23e31a3e0b)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:22.750478 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef6317-04a9-4b64-bf9a-42e42e123dfb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:22.795911 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-acef6317-04a9-4b64-bf9a-42e42e123dfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15e25d4ef32e4deab6401f23e31a3e0b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:22.797352 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-acef6317-04a9-4b64-bf9a-42e42e123dfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15e25d4ef32e4deab6401f23e31a3e0b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:22.797895 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-acef6317-04a9-4b64-bf9a-42e42e123dfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15e25d4ef32e4deab6401f23e31a3e0b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:22.804421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-acef6317-04a9-4b64-bf9a-42e42e123dfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15e25d4ef32e4deab6401f23e31a3e0b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:22.811515 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-acef6317-04a9-4b64-bf9a-42e42e123dfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15e25d4ef32e4deab6401f23e31a3e0b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:22.831094 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1266/2517] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:06:22 2022] DELETE /identity/v3/users/15e25d4ef32e4deab6401f23e31a3e0b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:22.835578 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ed23-6a52-4424-90db-d2ce449f82d6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:22.836050 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ed23-6a52-4424-90db-d2ce449f82d6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:22.836168 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ed23-6a52-4424-90db-d2ce449f82d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:22.854892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1743ed23-6a52-4424-90db-d2ce449f82d6 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:22.856099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1743ed23-6a52-4424-90db-d2ce449f82d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:22.918124 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1252/2518] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:22.928639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef6317-04a9-4b64-bf9a-42e42e123dfb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:22.930041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-acef6317-04a9-4b64-bf9a-42e42e123dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:22.992320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef6317-04a9-4b64-bf9a-42e42e123dfb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:22.996049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd5a45-5874-4107-b539-f0f57e187359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:22.996699 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd5a45-5874-4107-b539-f0f57e187359 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:22.996990 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd5a45-5874-4107-b539-f0f57e187359 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:22.997276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd5a45-5874-4107-b539-f0f57e187359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:23.007609 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd5a45-5874-4107-b539-f0f57e187359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:23.009099 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd5a45-5874-4107-b539-f0f57e187359 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:23.019897 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1267/2519] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:23.303752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743ed23-6a52-4424-90db-d2ce449f82d6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:23.305298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1743ed23-6a52-4424-90db-d2ce449f82d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:23.325173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743ed23-6a52-4424-90db-d2ce449f82d6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:23.331795 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfbd05-0999-45f8-80b6-d1516f61c7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:23.332984 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfbd05-0999-45f8-80b6-d1516f61c7af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:23.333457 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfbd05-0999-45f8-80b6-d1516f61c7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:23.333637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfbd05-0999-45f8-80b6-d1516f61c7af admin admin] PATH_INFO: `/v3/projects/423afd79d5564aea82943b0cd93dd128` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:23.335801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bfbd05-0999-45f8-80b6-d1516f61c7af admin admin] RBAC: Authorizing `identity:delete_project(project_id=423afd79d5564aea82943b0cd93dd128)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:23.338086 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bfbd05-0999-45f8-80b6-d1516f61c7af admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:23.392305 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-13bfbd05-0999-45f8-80b6-d1516f61c7af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '423afd79d5564aea82943b0cd93dd128', 'request_id': 'req-13bfbd05-0999-45f8-80b6-d1516f61c7af'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:23.403424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-13bfbd05-0999-45f8-80b6-d1516f61c7af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '423afd79d5564aea82943b0cd93dd128', 'request_id': 'req-13bfbd05-0999-45f8-80b6-d1516f61c7af'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:23.412191 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-13bfbd05-0999-45f8-80b6-d1516f61c7af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '423afd79d5564aea82943b0cd93dd128', 'request_id': 'req-13bfbd05-0999-45f8-80b6-d1516f61c7af'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:23.419462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:23.424927 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-13bfbd05-0999-45f8-80b6-d1516f61c7af admin admin] The token cache is being invalidate because project 423afd79d5564aea82943b0cd93dd128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:06:23.425623 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-13bfbd05-0999-45f8-80b6-d1516f61c7af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:23.426492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-13bfbd05-0999-45f8-80b6-d1516f61c7af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:23.428650 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1253/2520] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:06:23 2022] DELETE /identity/v3/projects/423afd79d5564aea82943b0cd93dd128 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:23.432365 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd5a45-5874-4107-b539-f0f57e187359 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:23.442903 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd5a45-5874-4107-b539-f0f57e187359 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:23.460965 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:23.506633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607cdf0-4297-4188-85f8-1589bb4f96a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:23.507528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607cdf0-4297-4188-85f8-1589bb4f96a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:23.507839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607cdf0-4297-4188-85f8-1589bb4f96a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:23.508645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607cdf0-4297-4188-85f8-1589bb4f96a2 admin admin] PATH_INFO: `/v3/users/e0b7cad2f83e4374862208463e21268d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:23.509382 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607cdf0-4297-4188-85f8-1589bb4f96a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0b7cad2f83e4374862208463e21268d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:23.511387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607cdf0-4297-4188-85f8-1589bb4f96a2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:23.562756 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4607cdf0-4297-4188-85f8-1589bb4f96a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0b7cad2f83e4374862208463e21268d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:23.563495 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4607cdf0-4297-4188-85f8-1589bb4f96a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0b7cad2f83e4374862208463e21268d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:23.564115 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4607cdf0-4297-4188-85f8-1589bb4f96a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0b7cad2f83e4374862208463e21268d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:23.573123 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4607cdf0-4297-4188-85f8-1589bb4f96a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0b7cad2f83e4374862208463e21268d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:23.581769 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4607cdf0-4297-4188-85f8-1589bb4f96a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0b7cad2f83e4374862208463e21268d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:23.604759 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1268/2521] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:06:23 2022] DELETE /identity/v3/users/e0b7cad2f83e4374862208463e21268d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:23.910671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfbd05-0999-45f8-80b6-d1516f61c7af admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:23.912206 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-13bfbd05-0999-45f8-80b6-d1516f61c7af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:23.933874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfbd05-0999-45f8-80b6-d1516f61c7af admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:24.017959 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12e74e-cbd4-498c-89f7-990e68b366b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:24.018873 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12e74e-cbd4-498c-89f7-990e68b366b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.019164 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12e74e-cbd4-498c-89f7-990e68b366b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.019444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12e74e-cbd4-498c-89f7-990e68b366b0 admin admin] PATH_INFO: `/v3/projects/1e556449cbfb46bf92c5fe52a914d03a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.020665 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12e74e-cbd4-498c-89f7-990e68b366b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e556449cbfb46bf92c5fe52a914d03a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:24.022597 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12e74e-cbd4-498c-89f7-990e68b366b0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:24.077269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-da12e74e-cbd4-498c-89f7-990e68b366b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e556449cbfb46bf92c5fe52a914d03a', 'request_id': 'req-da12e74e-cbd4-498c-89f7-990e68b366b0'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:24.086408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-da12e74e-cbd4-498c-89f7-990e68b366b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e556449cbfb46bf92c5fe52a914d03a', 'request_id': 'req-da12e74e-cbd4-498c-89f7-990e68b366b0'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:24.094281 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-da12e74e-cbd4-498c-89f7-990e68b366b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e556449cbfb46bf92c5fe52a914d03a', 'request_id': 'req-da12e74e-cbd4-498c-89f7-990e68b366b0'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:24.106219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-da12e74e-cbd4-498c-89f7-990e68b366b0 admin admin] The token cache is being invalidate because project 1e556449cbfb46bf92c5fe52a914d03a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:06:24.106685 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-da12e74e-cbd4-498c-89f7-990e68b366b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:24.107396 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-da12e74e-cbd4-498c-89f7-990e68b366b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:24.110116 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1254/2522] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:06:23 2022] DELETE /identity/v3/projects/1e556449cbfb46bf92c5fe52a914d03a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:24.115775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42088f8a-efef-4bb8-b474-df2f370c1246 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.116246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42088f8a-efef-4bb8-b474-df2f370c1246 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.116505 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42088f8a-efef-4bb8-b474-df2f370c1246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.137240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-42088f8a-efef-4bb8-b474-df2f370c1246 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:24.139055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-42088f8a-efef-4bb8-b474-df2f370c1246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:24.217676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1269/2523] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:24.227056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ebd53-a632-49be-93e6-ec4acb1580dd None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.227263 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ebd53-a632-49be-93e6-ec4acb1580dd None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.227547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ebd53-a632-49be-93e6-ec4acb1580dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.247660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-957ebd53-a632-49be-93e6-ec4acb1580dd None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:24.248896 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-957ebd53-a632-49be-93e6-ec4acb1580dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:24.259357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1255/2524] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:24.263871 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42088f8a-efef-4bb8-b474-df2f370c1246 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:24.273908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42088f8a-efef-4bb8-b474-df2f370c1246 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:24.279192 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2432f-5d23-400d-95a4-098bc4c4535b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:24.279868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2432f-5d23-400d-95a4-098bc4c4535b admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.280157 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2432f-5d23-400d-95a4-098bc4c4535b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.280456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2432f-5d23-400d-95a4-098bc4c4535b admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.281367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2432f-5d23-400d-95a4-098bc4c4535b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:06:24.281680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2432f-5d23-400d-95a4-098bc4c4535b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:24.283337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2432f-5d23-400d-95a4-098bc4c4535b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:24.296806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1270/2525] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:06:24.301296 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ebd53-a632-49be-93e6-ec4acb1580dd None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:24.311026 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ebd53-a632-49be-93e6-ec4acb1580dd None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:24.317239 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd6c4c-0093-46a3-884c-bef60f2f19cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:24.317998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd6c4c-0093-46a3-884c-bef60f2f19cf admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.318237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd6c4c-0093-46a3-884c-bef60f2f19cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.318511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd6c4c-0093-46a3-884c-bef60f2f19cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.319306 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd6c4c-0093-46a3-884c-bef60f2f19cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:24.321457 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd6c4c-0093-46a3-884c-bef60f2f19cf admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:24.349506 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1256/2526] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:06:24 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:06:24.353951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2432f-5d23-400d-95a4-098bc4c4535b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:24.365832 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2432f-5d23-400d-95a4-098bc4c4535b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:24.390993 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:24.429963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad467193-b8ff-40c2-ab1c-0307a6b05a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:24.430827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad467193-b8ff-40c2-ab1c-0307a6b05a82 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.431053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad467193-b8ff-40c2-ab1c-0307a6b05a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.431337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad467193-b8ff-40c2-ab1c-0307a6b05a82 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.432026 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad467193-b8ff-40c2-ab1c-0307a6b05a82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:24.434145 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad467193-b8ff-40c2-ab1c-0307a6b05a82 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:24.460556 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1271/2527] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:06:24 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:06:24.461798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd6c4c-0093-46a3-884c-bef60f2f19cf admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:24.463039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb926732-fb7b-4719-a900-84844cead86f None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.463370 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb926732-fb7b-4719-a900-84844cead86f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.463717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb926732-fb7b-4719-a900-84844cead86f None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.465489 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1272/2528] 10.222.0.70 () {54 vars in 921 bytes} [Thu Jun 16 19:06:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:06:24.475088 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd6c4c-0093-46a3-884c-bef60f2f19cf admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:24.476031 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6367f-b46a-477d-b15b-c511760e0564 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.476422 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6367f-b46a-477d-b15b-c511760e0564 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.476713 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6367f-b46a-477d-b15b-c511760e0564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.480279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f18b0-c71f-4b5e-a506-ebb168e2f764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:24.481004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f18b0-c71f-4b5e-a506-ebb168e2f764 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.481402 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f18b0-c71f-4b5e-a506-ebb168e2f764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.481718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f18b0-c71f-4b5e-a506-ebb168e2f764 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.482505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f18b0-c71f-4b5e-a506-ebb168e2f764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:24.486634 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f18b0-c71f-4b5e-a506-ebb168e2f764 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:24.497074 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1257/2529] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:06:24.499643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-28b6367f-b46a-477d-b15b-c511760e0564 None None] MFA Rules not processed for user `8342da838fe4462b9de2817ad3ac4ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:24.501119 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-28b6367f-b46a-477d-b15b-c511760e0564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:24.501590 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f18b0-c71f-4b5e-a506-ebb168e2f764 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:24.512344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f18b0-c71f-4b5e-a506-ebb168e2f764 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:24.519490 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c12d8-9dcf-4c70-bc3f-a16950765089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:24.520425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c12d8-9dcf-4c70-bc3f-a16950765089 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.520425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c12d8-9dcf-4c70-bc3f-a16950765089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.520848 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c12d8-9dcf-4c70-bc3f-a16950765089 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.521438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c12d8-9dcf-4c70-bc3f-a16950765089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:24.523517 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c12d8-9dcf-4c70-bc3f-a16950765089 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:24.539831 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1258/2530] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:06:24.541142 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c12d8-9dcf-4c70-bc3f-a16950765089 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:24.554239 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c12d8-9dcf-4c70-bc3f-a16950765089 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:24.560087 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca0cf0-11a7-4b28-8a7d-ae2173aff4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:24.561099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca0cf0-11a7-4b28-8a7d-ae2173aff4b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.561502 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca0cf0-11a7-4b28-8a7d-ae2173aff4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.561902 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca0cf0-11a7-4b28-8a7d-ae2173aff4b3 admin admin] PATH_INFO: `/v3/projects/61fa63f27536431199d94a6de5cf8af6/users/aefec8af441e48d89e61dd5a9eb33f78/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.579268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca0cf0-11a7-4b28-8a7d-ae2173aff4b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61fa63f27536431199d94a6de5cf8af6, user_id=aefec8af441e48d89e61dd5a9eb33f78, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:24.582086 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca0cf0-11a7-4b28-8a7d-ae2173aff4b3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:24.587255 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1273/2531] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Jun 16 19:06:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:24.598165 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1259/2532] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:06:24 2022] PUT /identity/v3/projects/61fa63f27536431199d94a6de5cf8af6/users/aefec8af441e48d89e61dd5a9eb33f78/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:06:24.608750 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fce036-1971-46a3-b212-e243094700aa None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.609550 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fce036-1971-46a3-b212-e243094700aa None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.609835 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fce036-1971-46a3-b212-e243094700aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.611579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca0cf0-11a7-4b28-8a7d-ae2173aff4b3 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:24.613410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e7ca0cf0-11a7-4b28-8a7d-ae2173aff4b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:24.628571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:24.638177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-13fce036-1971-46a3-b212-e243094700aa None None] MFA Rules not processed for user `aefec8af441e48d89e61dd5a9eb33f78`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:24.639580 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-13fce036-1971-46a3-b212-e243094700aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:24.678462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca0cf0-11a7-4b28-8a7d-ae2173aff4b3 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:24.682315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b054ea-f2bf-4d01-8d70-5df33863a344 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8563a25d16134d639a2a8849976fed9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:24.683364 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b054ea-f2bf-4d01-8d70-5df33863a344 service glance] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.683364 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b054ea-f2bf-4d01-8d70-5df33863a344 service glance] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.683599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b054ea-f2bf-4d01-8d70-5df33863a344 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.710955 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1274/2533] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:24.716409 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fce036-1971-46a3-b212-e243094700aa None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:24.758534 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b054ea-f2bf-4d01-8d70-5df33863a344 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:24.759995 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b054ea-f2bf-4d01-8d70-5df33863a344 service glance] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:24.763971 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:24.776663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1260/2534] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:24.797692 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fce036-1971-46a3-b212-e243094700aa None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:24.802390 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14442001-cc81-4b06-9d4a-a2a7d444f01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:24.803085 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14442001-cc81-4b06-9d4a-a2a7d444f01d admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.803371 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14442001-cc81-4b06-9d4a-a2a7d444f01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.803650 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14442001-cc81-4b06-9d4a-a2a7d444f01d admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.804450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14442001-cc81-4b06-9d4a-a2a7d444f01d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:24.806451 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14442001-cc81-4b06-9d4a-a2a7d444f01d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:24.840968 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1275/2535] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:06:24 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:06:24.842078 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b054ea-f2bf-4d01-8d70-5df33863a344 service glance] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:24.845908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a31c3a-7c82-4590-b6bd-9ef12b44ecba None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.846201 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a31c3a-7c82-4590-b6bd-9ef12b44ecba None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.846490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a31c3a-7c82-4590-b6bd-9ef12b44ecba None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.847831 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1276/2536] 10.222.0.70 () {54 vars in 921 bytes} [Thu Jun 16 19:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:06:24.852724 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4eb093-093b-4ca8-ab66-949f85aa308c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.853088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4eb093-093b-4ca8-ab66-949f85aa308c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.853366 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4eb093-093b-4ca8-ab66-949f85aa308c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.854442 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b054ea-f2bf-4d01-8d70-5df33863a344 service glance] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:24.872229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-cd4eb093-093b-4ca8-ab66-949f85aa308c None None] MFA Rules not processed for user `02a83f6e7aa448d6984a78914c1748cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:24.873508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cd4eb093-093b-4ca8-ab66-949f85aa308c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:24.915841 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00105072-dc30-4e43-bacb-d4906ea9623f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:24.916511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00105072-dc30-4e43-bacb-d4906ea9623f admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.916794 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00105072-dc30-4e43-bacb-d4906ea9623f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.917068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00105072-dc30-4e43-bacb-d4906ea9623f admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.917745 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00105072-dc30-4e43-bacb-d4906ea9623f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:24.919714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00105072-dc30-4e43-bacb-d4906ea9623f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:24.930978 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1277/2537] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Jun 16 19:06:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:06:24.942920 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4eb093-093b-4ca8-ab66-949f85aa308c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:24.943830 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1261/2538] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:06:24 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:06:24.944381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cd4eb093-093b-4ca8-ab66-949f85aa308c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:24.947600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00105072-dc30-4e43-bacb-d4906ea9623f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:24.956839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4eb093-093b-4ca8-ab66-949f85aa308c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:24.958617 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00105072-dc30-4e43-bacb-d4906ea9623f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:24.961038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f7ccd-4771-457a-bb8d-6678e098adda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:24.961630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f7ccd-4771-457a-bb8d-6678e098adda service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.961906 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f7ccd-4771-457a-bb8d-6678e098adda service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.962247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f7ccd-4771-457a-bb8d-6678e098adda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.965687 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04d416-4320-4f2a-9364-6e5c2cad52f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:24.966357 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04d416-4320-4f2a-9364-6e5c2cad52f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:24.966660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04d416-4320-4f2a-9364-6e5c2cad52f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:24.966935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04d416-4320-4f2a-9364-6e5c2cad52f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:24.967652 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04d416-4320-4f2a-9364-6e5c2cad52f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:24.970453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f7ccd-4771-457a-bb8d-6678e098adda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:24.970621 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04d416-4320-4f2a-9364-6e5c2cad52f4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:24.971897 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f7ccd-4771-457a-bb8d-6678e098adda service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:24.979715 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1278/2539] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:06:24.980935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1262/2540] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:06:24.985406 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f7ccd-4771-457a-bb8d-6678e098adda service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:24.996216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f7ccd-4771-457a-bb8d-6678e098adda service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:25.000922 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8da1a-57e3-4c6b-abad-b6a02e624374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:25.001797 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8da1a-57e3-4c6b-abad-b6a02e624374 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:25.002076 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8da1a-57e3-4c6b-abad-b6a02e624374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:25.002350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8da1a-57e3-4c6b-abad-b6a02e624374 admin admin] PATH_INFO: `/v3/projects/dcbdf9646d464ebc9206e1ccbd9e2679/users/e7f130edbc11429bb9f12a992a5ecfba/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:25.019216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8da1a-57e3-4c6b-abad-b6a02e624374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcbdf9646d464ebc9206e1ccbd9e2679, user_id=e7f130edbc11429bb9f12a992a5ecfba, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:25.021894 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8da1a-57e3-4c6b-abad-b6a02e624374 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:25.038005 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1279/2541] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:06:24 2022] PUT /identity/v3/projects/dcbdf9646d464ebc9206e1ccbd9e2679/users/e7f130edbc11429bb9f12a992a5ecfba/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:06:25.041836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04d416-4320-4f2a-9364-6e5c2cad52f4 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:25.053639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04d416-4320-4f2a-9364-6e5c2cad52f4 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:25.068443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c28a69-078b-4e6d-bc23-801689b1754e None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:25.068724 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c28a69-078b-4e6d-bc23-801689b1754e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:25.068998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c28a69-078b-4e6d-bc23-801689b1754e None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:25.070444 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1280/2542] 10.222.0.70 () {54 vars in 921 bytes} [Thu Jun 16 19:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:06:25.075974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec2c3b-dd49-4ffe-8ea8-c4c636d0ee2f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:25.076264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec2c3b-dd49-4ffe-8ea8-c4c636d0ee2f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:25.076566 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec2c3b-dd49-4ffe-8ea8-c4c636d0ee2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:25.088883 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:25.097635 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-54ec2c3b-dd49-4ffe-8ea8-c4c636d0ee2f None None] MFA Rules not processed for user `8342da838fe4462b9de2817ad3ac4ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:25.099118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-54ec2c3b-dd49-4ffe-8ea8-c4c636d0ee2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:25.119792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d1723-efd1-4b9d-be7d-725e3a8911e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:25.120491 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d1723-efd1-4b9d-be7d-725e3a8911e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:25.120778 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d1723-efd1-4b9d-be7d-725e3a8911e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:25.121095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d1723-efd1-4b9d-be7d-725e3a8911e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:25.121892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d1723-efd1-4b9d-be7d-725e3a8911e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:25.124716 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d1723-efd1-4b9d-be7d-725e3a8911e7 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:25.134601 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1263/2543] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:06:25.138788 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d1723-efd1-4b9d-be7d-725e3a8911e7 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:25.149458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d1723-efd1-4b9d-be7d-725e3a8911e7 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:25.154655 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02c965-2ee1-446c-ae60-4bf75633aa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:25.156199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02c965-2ee1-446c-ae60-4bf75633aa9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:25.156651 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02c965-2ee1-446c-ae60-4bf75633aa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:25.156941 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02c965-2ee1-446c-ae60-4bf75633aa9a admin admin] PATH_INFO: `/v3/projects/dcbdf9646d464ebc9206e1ccbd9e2679/users/e7f130edbc11429bb9f12a992a5ecfba/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:25.159004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02c965-2ee1-446c-ae60-4bf75633aa9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcbdf9646d464ebc9206e1ccbd9e2679, user_id=e7f130edbc11429bb9f12a992a5ecfba, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:25.161681 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02c965-2ee1-446c-ae60-4bf75633aa9a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:25.173676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1264/2544] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:06:25 2022] PUT /identity/v3/projects/dcbdf9646d464ebc9206e1ccbd9e2679/users/e7f130edbc11429bb9f12a992a5ecfba/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:06:25.178746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1281/2545] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Jun 16 19:06:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:06:25.184009 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb301b-6c9a-47f7-a8b0-d585e2868764 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:25.184304 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb301b-6c9a-47f7-a8b0-d585e2868764 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:25.184586 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb301b-6c9a-47f7-a8b0-d585e2868764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:25.189347 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec2c3b-dd49-4ffe-8ea8-c4c636d0ee2f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:25.212568 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-90cb301b-6c9a-47f7-a8b0-d585e2868764 None None] MFA Rules not processed for user `e7f130edbc11429bb9f12a992a5ecfba`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:25.213813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-90cb301b-6c9a-47f7-a8b0-d585e2868764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:25.220371 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:25.249579 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec2c3b-dd49-4ffe-8ea8-c4c636d0ee2f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:25.290615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1265/2546] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:06:25.305059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8a2bc-620c-45a6-9603-9b8d840b51fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:25.305518 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb301b-6c9a-47f7-a8b0-d585e2868764 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:25.306097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8a2bc-620c-45a6-9603-9b8d840b51fc service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:25.306480 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8a2bc-620c-45a6-9603-9b8d840b51fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:25.306859 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8a2bc-620c-45a6-9603-9b8d840b51fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:25.307509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-90cb301b-6c9a-47f7-a8b0-d585e2868764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:25.374017 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb301b-6c9a-47f7-a8b0-d585e2868764 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:25.379143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624e5b1-3e57-472b-be16-b1f6061cf505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:25.379798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624e5b1-3e57-472b-be16-b1f6061cf505 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:25.380084 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624e5b1-3e57-472b-be16-b1f6061cf505 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:25.380366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624e5b1-3e57-472b-be16-b1f6061cf505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:25.392674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624e5b1-3e57-472b-be16-b1f6061cf505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:25.393413 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d8a2bc-620c-45a6-9603-9b8d840b51fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:25.394138 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624e5b1-3e57-472b-be16-b1f6061cf505 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:25.395095 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d8a2bc-620c-45a6-9603-9b8d840b51fc service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:25.404791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1266/2547] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:25.407612 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1282/2548] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:26.022260 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624e5b1-3e57-472b-be16-b1f6061cf505 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:26.072210 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:26.090154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624e5b1-3e57-472b-be16-b1f6061cf505 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:26.095406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a32fd-9e64-4fee-8e94-a3562c4f4890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:26.096265 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a32fd-9e64-4fee-8e94-a3562c4f4890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:26.096543 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a32fd-9e64-4fee-8e94-a3562c4f4890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:26.096812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a32fd-9e64-4fee-8e94-a3562c4f4890 admin admin] PATH_INFO: `/v3/users/aefec8af441e48d89e61dd5a9eb33f78` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:26.098016 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a32fd-9e64-4fee-8e94-a3562c4f4890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aefec8af441e48d89e61dd5a9eb33f78)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:26.099976 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a32fd-9e64-4fee-8e94-a3562c4f4890 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:26.145355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-884a32fd-9e64-4fee-8e94-a3562c4f4890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aefec8af441e48d89e61dd5a9eb33f78'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:26.146008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-884a32fd-9e64-4fee-8e94-a3562c4f4890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aefec8af441e48d89e61dd5a9eb33f78'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:26.153140 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-884a32fd-9e64-4fee-8e94-a3562c4f4890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aefec8af441e48d89e61dd5a9eb33f78'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:26.165670 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-884a32fd-9e64-4fee-8e94-a3562c4f4890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aefec8af441e48d89e61dd5a9eb33f78'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:26.166308 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-884a32fd-9e64-4fee-8e94-a3562c4f4890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aefec8af441e48d89e61dd5a9eb33f78'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:26.180321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1267/2549] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:06:26 2022] DELETE /identity/v3/users/aefec8af441e48d89e61dd5a9eb33f78 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:26.184961 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf72d35-dd45-4b15-8b57-b1986078fcd6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:26.185420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf72d35-dd45-4b15-8b57-b1986078fcd6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:26.185564 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf72d35-dd45-4b15-8b57-b1986078fcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:26.203526 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-baf72d35-dd45-4b15-8b57-b1986078fcd6 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:26.204755 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-baf72d35-dd45-4b15-8b57-b1986078fcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:26.266604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1283/2550] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:26.276459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a32fd-9e64-4fee-8e94-a3562c4f4890 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:26.277926 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-884a32fd-9e64-4fee-8e94-a3562c4f4890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:26.346418 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a32fd-9e64-4fee-8e94-a3562c4f4890 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:26.354584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fcc49-2deb-4afd-b94c-2edbef0d6474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:26.355239 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fcc49-2deb-4afd-b94c-2edbef0d6474 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:26.355523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fcc49-2deb-4afd-b94c-2edbef0d6474 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:26.355799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fcc49-2deb-4afd-b94c-2edbef0d6474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:26.371044 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139fcc49-2deb-4afd-b94c-2edbef0d6474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:26.372490 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139fcc49-2deb-4afd-b94c-2edbef0d6474 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:26.382772 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1268/2551] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:26.994449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf72d35-dd45-4b15-8b57-b1986078fcd6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:26.996181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-baf72d35-dd45-4b15-8b57-b1986078fcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:27.010394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf72d35-dd45-4b15-8b57-b1986078fcd6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:27.015792 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924665f-c754-4c6e-89bf-cb6a702cb4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:27.016811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924665f-c754-4c6e-89bf-cb6a702cb4df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:27.017154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924665f-c754-4c6e-89bf-cb6a702cb4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:27.017539 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924665f-c754-4c6e-89bf-cb6a702cb4df admin admin] PATH_INFO: `/v3/projects/61fa63f27536431199d94a6de5cf8af6` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:27.018837 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924665f-c754-4c6e-89bf-cb6a702cb4df admin admin] RBAC: Authorizing `identity:delete_project(project_id=61fa63f27536431199d94a6de5cf8af6)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:27.020860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924665f-c754-4c6e-89bf-cb6a702cb4df admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:27.049999 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:27.078825 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b924665f-c754-4c6e-89bf-cb6a702cb4df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61fa63f27536431199d94a6de5cf8af6', 'request_id': 'req-b924665f-c754-4c6e-89bf-cb6a702cb4df'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:27.087874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b924665f-c754-4c6e-89bf-cb6a702cb4df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61fa63f27536431199d94a6de5cf8af6', 'request_id': 'req-b924665f-c754-4c6e-89bf-cb6a702cb4df'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:27.095054 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b924665f-c754-4c6e-89bf-cb6a702cb4df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61fa63f27536431199d94a6de5cf8af6', 'request_id': 'req-b924665f-c754-4c6e-89bf-cb6a702cb4df'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:27.109431 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b924665f-c754-4c6e-89bf-cb6a702cb4df admin admin] The token cache is being invalidate because project 61fa63f27536431199d94a6de5cf8af6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:06:27.109948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b924665f-c754-4c6e-89bf-cb6a702cb4df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:27.110852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b924665f-c754-4c6e-89bf-cb6a702cb4df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:27.115374 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1284/2552] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:06:26 2022] DELETE /identity/v3/projects/61fa63f27536431199d94a6de5cf8af6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:27.120500 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fcc49-2deb-4afd-b94c-2edbef0d6474 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:27.131518 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fcc49-2deb-4afd-b94c-2edbef0d6474 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:27.193040 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b999e-f941-4a9f-a291-6e36a58a2425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:27.193967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b999e-f941-4a9f-a291-6e36a58a2425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:27.194301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b999e-f941-4a9f-a291-6e36a58a2425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:27.194551 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b999e-f941-4a9f-a291-6e36a58a2425 admin admin] PATH_INFO: `/v3/users/e7f130edbc11429bb9f12a992a5ecfba` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:27.196171 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b999e-f941-4a9f-a291-6e36a58a2425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f130edbc11429bb9f12a992a5ecfba)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:27.198455 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b999e-f941-4a9f-a291-6e36a58a2425 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:27.205779 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:27.253459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b36b999e-f941-4a9f-a291-6e36a58a2425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f130edbc11429bb9f12a992a5ecfba'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:27.254868 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b36b999e-f941-4a9f-a291-6e36a58a2425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f130edbc11429bb9f12a992a5ecfba'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:27.262475 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b36b999e-f941-4a9f-a291-6e36a58a2425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f130edbc11429bb9f12a992a5ecfba'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:27.275837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b36b999e-f941-4a9f-a291-6e36a58a2425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f130edbc11429bb9f12a992a5ecfba'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:27.276577 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b36b999e-f941-4a9f-a291-6e36a58a2425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f130edbc11429bb9f12a992a5ecfba'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:27.291896 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1269/2553] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:06:27 2022] DELETE /identity/v3/users/e7f130edbc11429bb9f12a992a5ecfba => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:27.454828 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b255e6-58ed-4a4d-b45c-f63d59be2b92 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:27.455328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b255e6-58ed-4a4d-b45c-f63d59be2b92 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:27.455448 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b255e6-58ed-4a4d-b45c-f63d59be2b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:27.473073 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a3b255e6-58ed-4a4d-b45c-f63d59be2b92 None None] MFA Rules not processed for user `02c826fc0baa480393df4c884c9e35dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:27.474270 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a3b255e6-58ed-4a4d-b45c-f63d59be2b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:27.533480 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1285/2554] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Jun 16 19:06:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:27.545910 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b999e-f941-4a9f-a291-6e36a58a2425 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:27.547624 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b36b999e-f941-4a9f-a291-6e36a58a2425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:27.568386 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b255e6-58ed-4a4d-b45c-f63d59be2b92 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:27.570045 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a3b255e6-58ed-4a4d-b45c-f63d59be2b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:27.628142 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b999e-f941-4a9f-a291-6e36a58a2425 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:27.633999 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9df7d-dd6b-46f1-8250-38168e16ed0b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b22fe0ab0354385924923d0d443acae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:27.634754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9df7d-dd6b-46f1-8250-38168e16ed0b service swift] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:27.635110 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9df7d-dd6b-46f1-8250-38168e16ed0b service swift] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:27.635457 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9df7d-dd6b-46f1-8250-38168e16ed0b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:27.644972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9df7d-dd6b-46f1-8250-38168e16ed0b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:27.645149 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b255e6-58ed-4a4d-b45c-f63d59be2b92 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:27.646711 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9df7d-dd6b-46f1-8250-38168e16ed0b service swift] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:27.649971 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90952556-20ea-4b9a-8565-99e67e3f7cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:27.650950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90952556-20ea-4b9a-8565-99e67e3f7cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:27.651229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90952556-20ea-4b9a-8565-99e67e3f7cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:27.651925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90952556-20ea-4b9a-8565-99e67e3f7cc6 admin admin] PATH_INFO: `/v3/projects/dcbdf9646d464ebc9206e1ccbd9e2679` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:27.652720 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90952556-20ea-4b9a-8565-99e67e3f7cc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcbdf9646d464ebc9206e1ccbd9e2679)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:27.654741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90952556-20ea-4b9a-8565-99e67e3f7cc6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:27.656621 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1270/2555] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Jun 16 19:06:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:06:27.707786 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-90952556-20ea-4b9a-8565-99e67e3f7cc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcbdf9646d464ebc9206e1ccbd9e2679', 'request_id': 'req-90952556-20ea-4b9a-8565-99e67e3f7cc6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:27.711794 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:27.717511 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-90952556-20ea-4b9a-8565-99e67e3f7cc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcbdf9646d464ebc9206e1ccbd9e2679', 'request_id': 'req-90952556-20ea-4b9a-8565-99e67e3f7cc6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:27.743955 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-90952556-20ea-4b9a-8565-99e67e3f7cc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcbdf9646d464ebc9206e1ccbd9e2679', 'request_id': 'req-90952556-20ea-4b9a-8565-99e67e3f7cc6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:27.765182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-90952556-20ea-4b9a-8565-99e67e3f7cc6 admin admin] The token cache is being invalidate because project dcbdf9646d464ebc9206e1ccbd9e2679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:06:27.766302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-90952556-20ea-4b9a-8565-99e67e3f7cc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:27.767508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-90952556-20ea-4b9a-8565-99e67e3f7cc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:27.776415 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857c9bc-1ea2-4c06-9250-c33bc12b1154 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:27.776621 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857c9bc-1ea2-4c06-9250-c33bc12b1154 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:27.777130 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857c9bc-1ea2-4c06-9250-c33bc12b1154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:27.779313 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1286/2556] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:06:27 2022] DELETE /identity/v3/projects/dcbdf9646d464ebc9206e1ccbd9e2679 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:06:27.814692 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e857c9bc-1ea2-4c06-9250-c33bc12b1154 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:27.818441 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e857c9bc-1ea2-4c06-9250-c33bc12b1154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:27.940075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f20b1-a08d-4ad8-9e6d-986cb0d6c37b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:27.940704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f20b1-a08d-4ad8-9e6d-986cb0d6c37b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:27.940994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f20b1-a08d-4ad8-9e6d-986cb0d6c37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:27.947940 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1271/2557] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:27.965807 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8e9f20b1-a08d-4ad8-9e6d-986cb0d6c37b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:27.967075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8e9f20b1-a08d-4ad8-9e6d-986cb0d6c37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:27.980917 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1287/2558] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:27.985964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857c9bc-1ea2-4c06-9250-c33bc12b1154 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:28.006104 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857c9bc-1ea2-4c06-9250-c33bc12b1154 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:28.011493 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbde19f-1a1a-42ab-a8d5-754b61fb49bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:28.014714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbde19f-1a1a-42ab-a8d5-754b61fb49bb admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:28.014714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbde19f-1a1a-42ab-a8d5-754b61fb49bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:28.014714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbde19f-1a1a-42ab-a8d5-754b61fb49bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:28.014714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbde19f-1a1a-42ab-a8d5-754b61fb49bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:06:28.014714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbde19f-1a1a-42ab-a8d5-754b61fb49bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:28.017133 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbde19f-1a1a-42ab-a8d5-754b61fb49bb admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:28.036856 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f20b1-a08d-4ad8-9e6d-986cb0d6c37b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:28.042726 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1272/2559] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:06:28.048288 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f20b1-a08d-4ad8-9e6d-986cb0d6c37b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:28.056838 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289aacc-5a41-4acc-b89f-4d2b8c58a40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:28.057523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289aacc-5a41-4acc-b89f-4d2b8c58a40f admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:28.057809 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289aacc-5a41-4acc-b89f-4d2b8c58a40f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:28.058082 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289aacc-5a41-4acc-b89f-4d2b8c58a40f admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:28.061024 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289aacc-5a41-4acc-b89f-4d2b8c58a40f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:28.064505 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289aacc-5a41-4acc-b89f-4d2b8c58a40f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:28.102160 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1288/2560] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:06:28 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:06:28.105893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbde19f-1a1a-42ab-a8d5-754b61fb49bb admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:28.111435 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:28.117611 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbde19f-1a1a-42ab-a8d5-754b61fb49bb admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:28.188429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc599f-dadb-4fa7-8502-c676d1987d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:28.189280 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc599f-dadb-4fa7-8502-c676d1987d04 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:28.189658 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc599f-dadb-4fa7-8502-c676d1987d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:28.190026 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc599f-dadb-4fa7-8502-c676d1987d04 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:28.190843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc599f-dadb-4fa7-8502-c676d1987d04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:28.195386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc599f-dadb-4fa7-8502-c676d1987d04 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:28.214604 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1273/2561] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:06:28 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:06:28.218240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289aacc-5a41-4acc-b89f-4d2b8c58a40f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:28.230848 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289aacc-5a41-4acc-b89f-4d2b8c58a40f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:28.235808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50d4f1-6fee-4481-b7db-595d1f1afed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:28.239177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50d4f1-6fee-4481-b7db-595d1f1afed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:28.239666 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50d4f1-6fee-4481-b7db-595d1f1afed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:28.239941 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50d4f1-6fee-4481-b7db-595d1f1afed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:28.240676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50d4f1-6fee-4481-b7db-595d1f1afed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:28.242283 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50d4f1-6fee-4481-b7db-595d1f1afed6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:28.259973 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1289/2562] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:06:28.261298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc599f-dadb-4fa7-8502-c676d1987d04 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:28.274791 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc599f-dadb-4fa7-8502-c676d1987d04 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:28.280389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd6de4-7f5a-4486-a31c-a5236ebb3874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:28.282046 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd6de4-7f5a-4486-a31c-a5236ebb3874 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:28.282821 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd6de4-7f5a-4486-a31c-a5236ebb3874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:28.283182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd6de4-7f5a-4486-a31c-a5236ebb3874 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:28.284013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd6de4-7f5a-4486-a31c-a5236ebb3874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:28.285786 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd6de4-7f5a-4486-a31c-a5236ebb3874 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:28.299248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1274/2563] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:06:28.303248 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50d4f1-6fee-4481-b7db-595d1f1afed6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:28.314946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50d4f1-6fee-4481-b7db-595d1f1afed6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:28.321213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a6501-5235-41d4-8c78-962abac2fb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:28.322295 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a6501-5235-41d4-8c78-962abac2fb6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:28.322688 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a6501-5235-41d4-8c78-962abac2fb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:28.323042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a6501-5235-41d4-8c78-962abac2fb6a admin admin] PATH_INFO: `/v3/projects/f7b8164ebebf4730b4193c8aff908695/users/9da57a6863c142da855e1b98dc8b36e6/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:28.340394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a6501-5235-41d4-8c78-962abac2fb6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b8164ebebf4730b4193c8aff908695, user_id=9da57a6863c142da855e1b98dc8b36e6, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:28.343281 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a6501-5235-41d4-8c78-962abac2fb6a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:28.361112 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1290/2564] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:06:28 2022] PUT /identity/v3/projects/f7b8164ebebf4730b4193c8aff908695/users/9da57a6863c142da855e1b98dc8b36e6/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:06:28.371824 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae76b3e-98df-42f4-8fbb-01e86a12f69a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:28.372117 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae76b3e-98df-42f4-8fbb-01e86a12f69a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:28.372401 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae76b3e-98df-42f4-8fbb-01e86a12f69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:28.403616 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-eae76b3e-98df-42f4-8fbb-01e86a12f69a None None] MFA Rules not processed for user `9da57a6863c142da855e1b98dc8b36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:28.404971 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-eae76b3e-98df-42f4-8fbb-01e86a12f69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:28.479631 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1275/2565] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:28.492175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a6501-5235-41d4-8c78-962abac2fb6a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:28.493873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7f7a6501-5235-41d4-8c78-962abac2fb6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:28.555842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a6501-5235-41d4-8c78-962abac2fb6a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:28.559617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293aeeb-4fca-40ad-820c-67e9bdb54184 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:28.560265 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293aeeb-4fca-40ad-820c-67e9bdb54184 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:28.560550 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293aeeb-4fca-40ad-820c-67e9bdb54184 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:28.560823 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293aeeb-4fca-40ad-820c-67e9bdb54184 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:28.572237 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293aeeb-4fca-40ad-820c-67e9bdb54184 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:28.573681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293aeeb-4fca-40ad-820c-67e9bdb54184 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:28.584109 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1291/2566] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:28.604264 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae76b3e-98df-42f4-8fbb-01e86a12f69a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:28.673889 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae76b3e-98df-42f4-8fbb-01e86a12f69a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:28.678515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31842c-c353-42be-9781-53182f9c6e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:28.679525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31842c-c353-42be-9781-53182f9c6e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:28.679832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31842c-c353-42be-9781-53182f9c6e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:28.680117 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31842c-c353-42be-9781-53182f9c6e23 admin admin] PATH_INFO: `/v3/users/9da57a6863c142da855e1b98dc8b36e6` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:28.681386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31842c-c353-42be-9781-53182f9c6e23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9da57a6863c142da855e1b98dc8b36e6)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:28.683504 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31842c-c353-42be-9781-53182f9c6e23 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:28.699105 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:28.744035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-dd31842c-c353-42be-9781-53182f9c6e23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9da57a6863c142da855e1b98dc8b36e6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:28.744776 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-dd31842c-c353-42be-9781-53182f9c6e23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9da57a6863c142da855e1b98dc8b36e6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:28.760670 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-dd31842c-c353-42be-9781-53182f9c6e23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9da57a6863c142da855e1b98dc8b36e6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:28.776741 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-dd31842c-c353-42be-9781-53182f9c6e23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9da57a6863c142da855e1b98dc8b36e6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:28.777556 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-dd31842c-c353-42be-9781-53182f9c6e23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9da57a6863c142da855e1b98dc8b36e6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:28.793894 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1276/2567] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:06:28 2022] DELETE /identity/v3/users/9da57a6863c142da855e1b98dc8b36e6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:28.798719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fdd1d-5312-461a-934d-18217161033e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:28.799011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fdd1d-5312-461a-934d-18217161033e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:28.799319 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fdd1d-5312-461a-934d-18217161033e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:28.817325 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3b3fdd1d-5312-461a-934d-18217161033e None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:28.821781 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3b3fdd1d-5312-461a-934d-18217161033e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:28.888173 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1292/2568] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:28.900377 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31842c-c353-42be-9781-53182f9c6e23 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:28.902055 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-dd31842c-c353-42be-9781-53182f9c6e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:28.974504 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31842c-c353-42be-9781-53182f9c6e23 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:28.978589 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34afc2-f0b3-43c7-a679-a37d7de639f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:28.979432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34afc2-f0b3-43c7-a679-a37d7de639f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:28.979823 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34afc2-f0b3-43c7-a679-a37d7de639f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:28.980106 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34afc2-f0b3-43c7-a679-a37d7de639f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:28.990957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34afc2-f0b3-43c7-a679-a37d7de639f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:28.992414 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34afc2-f0b3-43c7-a679-a37d7de639f1 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:29.004093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1277/2569] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:29.313047 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fdd1d-5312-461a-934d-18217161033e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:29.314913 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3b3fdd1d-5312-461a-934d-18217161033e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:29.328559 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fdd1d-5312-461a-934d-18217161033e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:29.336358 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b878ea-77dd-4581-baef-43c2e70ee36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:29.337951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b878ea-77dd-4581-baef-43c2e70ee36e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:29.338469 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b878ea-77dd-4581-baef-43c2e70ee36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:29.338845 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b878ea-77dd-4581-baef-43c2e70ee36e admin admin] PATH_INFO: `/v3/projects/f7b8164ebebf4730b4193c8aff908695` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:29.340835 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b878ea-77dd-4581-baef-43c2e70ee36e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7b8164ebebf4730b4193c8aff908695)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:29.343573 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b878ea-77dd-4581-baef-43c2e70ee36e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:29.369598 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:29.455101 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c7b878ea-77dd-4581-baef-43c2e70ee36e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7b8164ebebf4730b4193c8aff908695', 'request_id': 'req-c7b878ea-77dd-4581-baef-43c2e70ee36e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:29.473806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c7b878ea-77dd-4581-baef-43c2e70ee36e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7b8164ebebf4730b4193c8aff908695', 'request_id': 'req-c7b878ea-77dd-4581-baef-43c2e70ee36e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:29.487643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c7b878ea-77dd-4581-baef-43c2e70ee36e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7b8164ebebf4730b4193c8aff908695', 'request_id': 'req-c7b878ea-77dd-4581-baef-43c2e70ee36e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:29.502260 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c7b878ea-77dd-4581-baef-43c2e70ee36e admin admin] The token cache is being invalidate because project f7b8164ebebf4730b4193c8aff908695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:06:29.503320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c7b878ea-77dd-4581-baef-43c2e70ee36e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:29.504234 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c7b878ea-77dd-4581-baef-43c2e70ee36e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:29.506320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1293/2570] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:06:29 2022] DELETE /identity/v3/projects/f7b8164ebebf4730b4193c8aff908695 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:29.511170 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdad91-6546-4a63-9143-a6fed1fa78c7 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:29.511619 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdad91-6546-4a63-9143-a6fed1fa78c7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:29.511702 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdad91-6546-4a63-9143-a6fed1fa78c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:29.552182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1abdad91-6546-4a63-9143-a6fed1fa78c7 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:29.553410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1abdad91-6546-4a63-9143-a6fed1fa78c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:29.674675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1278/2571] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:29.682833 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d131b60-afdd-4bb7-a10b-c24bf1d624ce None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:29.683156 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d131b60-afdd-4bb7-a10b-c24bf1d624ce None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:29.683445 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d131b60-afdd-4bb7-a10b-c24bf1d624ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:29.704349 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4d131b60-afdd-4bb7-a10b-c24bf1d624ce None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:29.705637 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4d131b60-afdd-4bb7-a10b-c24bf1d624ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:29.714199 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1294/2572] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:29.723940 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdad91-6546-4a63-9143-a6fed1fa78c7 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:29.738705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdad91-6546-4a63-9143-a6fed1fa78c7 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:29.743863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31694d-749a-4da7-98af-218d9b8a0712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:29.744528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31694d-749a-4da7-98af-218d9b8a0712 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:29.744823 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31694d-749a-4da7-98af-218d9b8a0712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:29.745103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31694d-749a-4da7-98af-218d9b8a0712 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:29.748598 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31694d-749a-4da7-98af-218d9b8a0712 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:06:29.748961 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31694d-749a-4da7-98af-218d9b8a0712 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:29.751750 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31694d-749a-4da7-98af-218d9b8a0712 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:29.769654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1279/2573] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:06:29.774413 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d131b60-afdd-4bb7-a10b-c24bf1d624ce None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:29.785096 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d131b60-afdd-4bb7-a10b-c24bf1d624ce None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:29.789969 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f28b8-5fb7-4974-9f40-484df3814a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:29.790676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f28b8-5fb7-4974-9f40-484df3814a01 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:29.790946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f28b8-5fb7-4974-9f40-484df3814a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:29.791218 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f28b8-5fb7-4974-9f40-484df3814a01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:29.792081 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f28b8-5fb7-4974-9f40-484df3814a01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:29.794248 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f28b8-5fb7-4974-9f40-484df3814a01 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:29.826610 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1295/2574] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:06:29 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:06:29.828778 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31694d-749a-4da7-98af-218d9b8a0712 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:29.840558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31694d-749a-4da7-98af-218d9b8a0712 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:29.861187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:29.907573 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf38ae-a6e3-4251-bc3a-025f3c6faab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:29.908295 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf38ae-a6e3-4251-bc3a-025f3c6faab4 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:29.908584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf38ae-a6e3-4251-bc3a-025f3c6faab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:29.908843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf38ae-a6e3-4251-bc3a-025f3c6faab4 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:29.909603 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf38ae-a6e3-4251-bc3a-025f3c6faab4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:29.911581 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf38ae-a6e3-4251-bc3a-025f3c6faab4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:29.938405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1280/2575] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:06:29 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:06:29.942148 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f28b8-5fb7-4974-9f40-484df3814a01 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:29.955649 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f28b8-5fb7-4974-9f40-484df3814a01 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:29.960874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3834e3-18c2-4f94-a521-aa4b0b434378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:29.961745 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3834e3-18c2-4f94-a521-aa4b0b434378 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:29.962179 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3834e3-18c2-4f94-a521-aa4b0b434378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:29.962679 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3834e3-18c2-4f94-a521-aa4b0b434378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:29.963410 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3834e3-18c2-4f94-a521-aa4b0b434378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:29.965013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3834e3-18c2-4f94-a521-aa4b0b434378 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:29.980073 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1296/2576] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:06:29.981247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf38ae-a6e3-4251-bc3a-025f3c6faab4 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:29.992971 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf38ae-a6e3-4251-bc3a-025f3c6faab4 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:29.998295 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf148c-2eb6-4305-97c1-56d81f0b5560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:29.999150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf148c-2eb6-4305-97c1-56d81f0b5560 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:30.000068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf148c-2eb6-4305-97c1-56d81f0b5560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:30.000834 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf148c-2eb6-4305-97c1-56d81f0b5560 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:30.001604 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf148c-2eb6-4305-97c1-56d81f0b5560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:30.003270 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf148c-2eb6-4305-97c1-56d81f0b5560 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:30.017351 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1281/2577] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:06:30.018789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3834e3-18c2-4f94-a521-aa4b0b434378 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:30.030011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3834e3-18c2-4f94-a521-aa4b0b434378 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:30.037097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793d387-02cc-41d0-a369-63bcde8919cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:30.038039 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793d387-02cc-41d0-a369-63bcde8919cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:30.038321 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793d387-02cc-41d0-a369-63bcde8919cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:30.038736 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793d387-02cc-41d0-a369-63bcde8919cf admin admin] PATH_INFO: `/v3/projects/5903716053d04af9b52b4c39c1100900/users/cd983e2908fc4733bd3ea8e296c36277/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:30.057695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793d387-02cc-41d0-a369-63bcde8919cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5903716053d04af9b52b4c39c1100900, user_id=cd983e2908fc4733bd3ea8e296c36277, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:30.060486 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793d387-02cc-41d0-a369-63bcde8919cf admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:30.076610 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1297/2578] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:06:30 2022] PUT /identity/v3/projects/5903716053d04af9b52b4c39c1100900/users/cd983e2908fc4733bd3ea8e296c36277/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:06:30.145585 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d11ea-fdf8-4272-9e31-82ad6079c3bf None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:30.145585 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d11ea-fdf8-4272-9e31-82ad6079c3bf None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:30.145585 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d11ea-fdf8-4272-9e31-82ad6079c3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:30.174691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b77d11ea-fdf8-4272-9e31-82ad6079c3bf None None] MFA Rules not processed for user `cd983e2908fc4733bd3ea8e296c36277`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:30.176073 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b77d11ea-fdf8-4272-9e31-82ad6079c3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:30.248737 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1282/2579] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:30.260570 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793d387-02cc-41d0-a369-63bcde8919cf admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:30.262393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b793d387-02cc-41d0-a369-63bcde8919cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:30.323953 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793d387-02cc-41d0-a369-63bcde8919cf admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:30.328187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a2a70-0b61-4acc-b7a5-0370dc6d0195 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:30.328793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a2a70-0b61-4acc-b7a5-0370dc6d0195 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:30.329073 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a2a70-0b61-4acc-b7a5-0370dc6d0195 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:30.329405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a2a70-0b61-4acc-b7a5-0370dc6d0195 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:30.338506 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a2a70-0b61-4acc-b7a5-0370dc6d0195 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:30.340143 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a2a70-0b61-4acc-b7a5-0370dc6d0195 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:30.350465 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1298/2580] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:38.595195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d11ea-fdf8-4272-9e31-82ad6079c3bf None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:38.660583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d11ea-fdf8-4272-9e31-82ad6079c3bf None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:38.665128 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00611139-7a40-43fa-9ed1-a6d00d4f139b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:38.666005 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00611139-7a40-43fa-9ed1-a6d00d4f139b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:38.666284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00611139-7a40-43fa-9ed1-a6d00d4f139b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:38.666561 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00611139-7a40-43fa-9ed1-a6d00d4f139b admin admin] PATH_INFO: `/v3/users/cd983e2908fc4733bd3ea8e296c36277` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:38.667806 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00611139-7a40-43fa-9ed1-a6d00d4f139b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd983e2908fc4733bd3ea8e296c36277)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:38.669721 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00611139-7a40-43fa-9ed1-a6d00d4f139b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:38.707234 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:38.725450 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-00611139-7a40-43fa-9ed1-a6d00d4f139b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd983e2908fc4733bd3ea8e296c36277'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:38.726217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-00611139-7a40-43fa-9ed1-a6d00d4f139b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd983e2908fc4733bd3ea8e296c36277'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:38.734050 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-00611139-7a40-43fa-9ed1-a6d00d4f139b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd983e2908fc4733bd3ea8e296c36277'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:38.745709 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-00611139-7a40-43fa-9ed1-a6d00d4f139b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd983e2908fc4733bd3ea8e296c36277'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:38.746349 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-00611139-7a40-43fa-9ed1-a6d00d4f139b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd983e2908fc4733bd3ea8e296c36277'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:38.760224 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1283/2581] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:06:38 2022] DELETE /identity/v3/users/cd983e2908fc4733bd3ea8e296c36277 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:38.764789 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d900796-a509-4a39-a9c7-49b577227bf4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:38.765274 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d900796-a509-4a39-a9c7-49b577227bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:38.765387 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d900796-a509-4a39-a9c7-49b577227bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:38.784430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1d900796-a509-4a39-a9c7-49b577227bf4 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:38.785686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1d900796-a509-4a39-a9c7-49b577227bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:38.833393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:38.854482 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1299/2582] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:38.864755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00611139-7a40-43fa-9ed1-a6d00d4f139b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:38.866156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-00611139-7a40-43fa-9ed1-a6d00d4f139b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:38.930359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00611139-7a40-43fa-9ed1-a6d00d4f139b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:38.934039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7833182-93ac-4287-af26-47ec88f0643e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:38.934740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7833182-93ac-4287-af26-47ec88f0643e service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:38.935019 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7833182-93ac-4287-af26-47ec88f0643e service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:38.935292 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7833182-93ac-4287-af26-47ec88f0643e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:38.944608 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7833182-93ac-4287-af26-47ec88f0643e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:38.946025 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7833182-93ac-4287-af26-47ec88f0643e service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:38.956552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1284/2583] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:39.244185 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d900796-a509-4a39-a9c7-49b577227bf4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:39.245786 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1d900796-a509-4a39-a9c7-49b577227bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:39.264931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d900796-a509-4a39-a9c7-49b577227bf4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:39.270861 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71630bea-ff89-4493-a52f-e3ec57f8da84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:39.271775 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71630bea-ff89-4493-a52f-e3ec57f8da84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:39.272058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71630bea-ff89-4493-a52f-e3ec57f8da84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:39.272338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71630bea-ff89-4493-a52f-e3ec57f8da84 admin admin] PATH_INFO: `/v3/projects/5903716053d04af9b52b4c39c1100900` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:39.273646 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71630bea-ff89-4493-a52f-e3ec57f8da84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5903716053d04af9b52b4c39c1100900)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:39.275938 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71630bea-ff89-4493-a52f-e3ec57f8da84 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:39.325521 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-71630bea-ff89-4493-a52f-e3ec57f8da84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5903716053d04af9b52b4c39c1100900', 'request_id': 'req-71630bea-ff89-4493-a52f-e3ec57f8da84'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:39.333321 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-71630bea-ff89-4493-a52f-e3ec57f8da84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5903716053d04af9b52b4c39c1100900', 'request_id': 'req-71630bea-ff89-4493-a52f-e3ec57f8da84'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:39.340473 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-71630bea-ff89-4493-a52f-e3ec57f8da84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5903716053d04af9b52b4c39c1100900', 'request_id': 'req-71630bea-ff89-4493-a52f-e3ec57f8da84'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:39.351758 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-71630bea-ff89-4493-a52f-e3ec57f8da84 admin admin] The token cache is being invalidate because project 5903716053d04af9b52b4c39c1100900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:06:39.352052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-71630bea-ff89-4493-a52f-e3ec57f8da84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:39.352704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-71630bea-ff89-4493-a52f-e3ec57f8da84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:39.354230 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1300/2584] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:06:39 2022] DELETE /identity/v3/projects/5903716053d04af9b52b4c39c1100900 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:43.347062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7833182-93ac-4287-af26-47ec88f0643e service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:43.355837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7833182-93ac-4287-af26-47ec88f0643e service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:43.408461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaddf58-1c10-4ebe-9882-7fb219999a88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:43.409130 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaddf58-1c10-4ebe-9882-7fb219999a88 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:43.409408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaddf58-1c10-4ebe-9882-7fb219999a88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:43.409679 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaddf58-1c10-4ebe-9882-7fb219999a88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:43.411548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-cfaddf58-1c10-4ebe-9882-7fb219999a88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:43.471219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaddf58-1c10-4ebe-9882-7fb219999a88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:43.472582 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaddf58-1c10-4ebe-9882-7fb219999a88 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:43.482301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1285/2585] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:43.718872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451c60a-cc32-4947-ad27-f65c3a797acb None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:43.720504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451c60a-cc32-4947-ad27-f65c3a797acb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:43.721054 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451c60a-cc32-4947-ad27-f65c3a797acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:43.746526 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9451c60a-cc32-4947-ad27-f65c3a797acb None None] MFA Rules not processed for user `02c826fc0baa480393df4c884c9e35dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:43.747928 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9451c60a-cc32-4947-ad27-f65c3a797acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:43.817537 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1301/2586] 10.222.0.70 () {58 vars in 967 bytes} [Thu Jun 16 19:06:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:43.833507 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaddf58-1c10-4ebe-9882-7fb219999a88 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:43.833507 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-cfaddf58-1c10-4ebe-9882-7fb219999a88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:43.841505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:43.930264 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaddf58-1c10-4ebe-9882-7fb219999a88 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:43.935766 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3323c-06a8-4ab9-b33e-03ccc369d13e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b22fe0ab0354385924923d0d443acae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:43.936775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3323c-06a8-4ab9-b33e-03ccc369d13e service swift] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:43.937195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3323c-06a8-4ab9-b33e-03ccc369d13e service swift] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:43.937454 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3323c-06a8-4ab9-b33e-03ccc369d13e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:43.947576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3323c-06a8-4ab9-b33e-03ccc369d13e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:43.950448 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3323c-06a8-4ab9-b33e-03ccc369d13e service swift] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:43.960488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1286/2587] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Jun 16 19:06:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:06:49.598627 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451c60a-cc32-4947-ad27-f65c3a797acb None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:49.600061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9451c60a-cc32-4947-ad27-f65c3a797acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:49.612468 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451c60a-cc32-4947-ad27-f65c3a797acb None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:49.616026 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9caa25-c333-414d-93c4-7b5e60b93ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:49.616664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9caa25-c333-414d-93c4-7b5e60b93ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:49.616943 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9caa25-c333-414d-93c4-7b5e60b93ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:49.617217 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9caa25-c333-414d-93c4-7b5e60b93ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:49.618980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8f9caa25-c333-414d-93c4-7b5e60b93ed8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:49.684027 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9caa25-c333-414d-93c4-7b5e60b93ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:49.685565 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9caa25-c333-414d-93c4-7b5e60b93ed8 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:49.695691 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1302/2588] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:53.000871 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3323c-06a8-4ab9-b33e-03ccc369d13e service swift] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:53.002804 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-84e3323c-06a8-4ab9-b33e-03ccc369d13e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:53.091198 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3323c-06a8-4ab9-b33e-03ccc369d13e service swift] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:53.101141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d3b28-ad22-451f-9153-29b25efd6794 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '880d0299c38c4eb897c669a512e599be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:53.101141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d3b28-ad22-451f-9153-29b25efd6794 service placement] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:53.101141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d3b28-ad22-451f-9153-29b25efd6794 service placement] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:53.101141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d3b28-ad22-451f-9153-29b25efd6794 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:53.101663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-924d3b28-ad22-451f-9153-29b25efd6794 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:53.113975 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d3b28-ad22-451f-9153-29b25efd6794 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:53.115753 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d3b28-ad22-451f-9153-29b25efd6794 service placement] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:53.128631 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1287/2589] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:56.632521 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9caa25-c333-414d-93c4-7b5e60b93ed8 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:56.634184 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8f9caa25-c333-414d-93c4-7b5e60b93ed8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:56.652248 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9caa25-c333-414d-93c4-7b5e60b93ed8 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:56.658135 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf18dc9-f072-457e-94d8-def65cb2e36c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '880d0299c38c4eb897c669a512e599be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:56.660136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf18dc9-f072-457e-94d8-def65cb2e36c service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:56.660446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf18dc9-f072-457e-94d8-def65cb2e36c service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:56.660717 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf18dc9-f072-457e-94d8-def65cb2e36c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:56.663026 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ccf18dc9-f072-457e-94d8-def65cb2e36c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:56.678594 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:56.687467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf18dc9-f072-457e-94d8-def65cb2e36c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:56.689363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf18dc9-f072-457e-94d8-def65cb2e36c service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:56.706181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1303/2590] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:56.715841 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d3b28-ad22-451f-9153-29b25efd6794 service placement] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:56.718184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-924d3b28-ad22-451f-9153-29b25efd6794 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:56.737100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d3b28-ad22-451f-9153-29b25efd6794 service placement] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:56.747882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5be38d-d6d0-4471-ad11-9d73ebabe4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:56.748907 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5be38d-d6d0-4471-ad11-9d73ebabe4b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:56.749269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5be38d-d6d0-4471-ad11-9d73ebabe4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:56.749547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5be38d-d6d0-4471-ad11-9d73ebabe4b6 admin admin] PATH_INFO: `/v3/users/27db09b804fa426a83f103cf5dd116a8` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:56.751142 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5be38d-d6d0-4471-ad11-9d73ebabe4b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27db09b804fa426a83f103cf5dd116a8)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:56.753331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5be38d-d6d0-4471-ad11-9d73ebabe4b6 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:56.806158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2f5be38d-d6d0-4471-ad11-9d73ebabe4b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27db09b804fa426a83f103cf5dd116a8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:56.806857 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2f5be38d-d6d0-4471-ad11-9d73ebabe4b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27db09b804fa426a83f103cf5dd116a8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:56.814762 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2f5be38d-d6d0-4471-ad11-9d73ebabe4b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27db09b804fa426a83f103cf5dd116a8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:56.832290 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2f5be38d-d6d0-4471-ad11-9d73ebabe4b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27db09b804fa426a83f103cf5dd116a8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:56.833042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2f5be38d-d6d0-4471-ad11-9d73ebabe4b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27db09b804fa426a83f103cf5dd116a8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:56.837150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:56.848534 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1288/2591] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:06:56 2022] DELETE /identity/v3/users/27db09b804fa426a83f103cf5dd116a8 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:57.036165 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf18dc9-f072-457e-94d8-def65cb2e36c service placement] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:57.051176 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf18dc9-f072-457e-94d8-def65cb2e36c service placement] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:57.109584 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:57.110446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:57.110851 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:57.111132 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37 admin admin] PATH_INFO: `/v3/projects/629634dc3dc54893b68cb9c9dc08505d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:57.112304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=629634dc3dc54893b68cb9c9dc08505d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:57.114346 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:57.161599 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '629634dc3dc54893b68cb9c9dc08505d', 'request_id': 'req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:57.169217 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '629634dc3dc54893b68cb9c9dc08505d', 'request_id': 'req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:57.175869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '629634dc3dc54893b68cb9c9dc08505d', 'request_id': 'req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:57.186800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37 admin admin] The token cache is being invalidate because project 629634dc3dc54893b68cb9c9dc08505d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:06:57.187088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:57.187726 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:57.189278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1304/2592] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:06:57 2022] DELETE /identity/v3/projects/629634dc3dc54893b68cb9c9dc08505d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:57.193235 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5be38d-d6d0-4471-ad11-9d73ebabe4b6 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:57.194725 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2f5be38d-d6d0-4471-ad11-9d73ebabe4b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:57.208847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5be38d-d6d0-4471-ad11-9d73ebabe4b6 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:57.277226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fa8db-5b53-4c6c-a360-2b324e0445c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:57.277226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fa8db-5b53-4c6c-a360-2b324e0445c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:57.277226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fa8db-5b53-4c6c-a360-2b324e0445c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:57.277226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fa8db-5b53-4c6c-a360-2b324e0445c1 admin admin] PATH_INFO: `/v3/users/6528d33ac137459facf75fa72e70cf9c` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:57.279007 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0fa8db-5b53-4c6c-a360-2b324e0445c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6528d33ac137459facf75fa72e70cf9c)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:57.279883 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0fa8db-5b53-4c6c-a360-2b324e0445c1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:57.344611 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-0a0fa8db-5b53-4c6c-a360-2b324e0445c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6528d33ac137459facf75fa72e70cf9c'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:57.345302 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-0a0fa8db-5b53-4c6c-a360-2b324e0445c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6528d33ac137459facf75fa72e70cf9c'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:57.353565 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-0a0fa8db-5b53-4c6c-a360-2b324e0445c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6528d33ac137459facf75fa72e70cf9c'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:57.368321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-0a0fa8db-5b53-4c6c-a360-2b324e0445c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6528d33ac137459facf75fa72e70cf9c'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:57.369045 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-0a0fa8db-5b53-4c6c-a360-2b324e0445c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6528d33ac137459facf75fa72e70cf9c'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:57.385703 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1289/2593] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:06:57 2022] DELETE /identity/v3/users/6528d33ac137459facf75fa72e70cf9c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:57.634023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:57.635825 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:57.652596 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1aeae-08d7-4165-a37c-d48cf0d8ba37 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:57.715754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345da2b6-de11-4f1a-92ef-34dcead5488d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:57.717280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345da2b6-de11-4f1a-92ef-34dcead5488d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:57.718211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345da2b6-de11-4f1a-92ef-34dcead5488d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:57.718497 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345da2b6-de11-4f1a-92ef-34dcead5488d admin admin] PATH_INFO: `/v3/projects/59a97968d23943bba34843c82aa0b44d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:57.719896 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345da2b6-de11-4f1a-92ef-34dcead5488d admin admin] RBAC: Authorizing `identity:delete_project(project_id=59a97968d23943bba34843c82aa0b44d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:57.722007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345da2b6-de11-4f1a-92ef-34dcead5488d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:57.757253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:57.776931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-345da2b6-de11-4f1a-92ef-34dcead5488d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59a97968d23943bba34843c82aa0b44d', 'request_id': 'req-345da2b6-de11-4f1a-92ef-34dcead5488d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:57.785216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-345da2b6-de11-4f1a-92ef-34dcead5488d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59a97968d23943bba34843c82aa0b44d', 'request_id': 'req-345da2b6-de11-4f1a-92ef-34dcead5488d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:57.791870 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-345da2b6-de11-4f1a-92ef-34dcead5488d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59a97968d23943bba34843c82aa0b44d', 'request_id': 'req-345da2b6-de11-4f1a-92ef-34dcead5488d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:57.803710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-345da2b6-de11-4f1a-92ef-34dcead5488d admin admin] The token cache is being invalidate because project 59a97968d23943bba34843c82aa0b44d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:06:57.803998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-345da2b6-de11-4f1a-92ef-34dcead5488d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:57.804736 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-345da2b6-de11-4f1a-92ef-34dcead5488d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:57.806363 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1305/2594] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:06:57 2022] DELETE /identity/v3/projects/59a97968d23943bba34843c82aa0b44d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:01.596920 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fa8db-5b53-4c6c-a360-2b324e0445c1 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:01.598316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0a0fa8db-5b53-4c6c-a360-2b324e0445c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:01.656779 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fa8db-5b53-4c6c-a360-2b324e0445c1 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:01.685965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:01.708441 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f86a1f-4caa-4530-86a5-0ecd19b69549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:01.709090 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f86a1f-4caa-4530-86a5-0ecd19b69549 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:01.709391 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f86a1f-4caa-4530-86a5-0ecd19b69549 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:01.709661 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f86a1f-4caa-4530-86a5-0ecd19b69549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:01.711538 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-61f86a1f-4caa-4530-86a5-0ecd19b69549 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:01.774775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f86a1f-4caa-4530-86a5-0ecd19b69549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:01.776142 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f86a1f-4caa-4530-86a5-0ecd19b69549 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:01.786379 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1290/2595] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:05.019417 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345da2b6-de11-4f1a-92ef-34dcead5488d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:05.020864 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-345da2b6-de11-4f1a-92ef-34dcead5488d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:05.036305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345da2b6-de11-4f1a-92ef-34dcead5488d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:05.042710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d6a97-9a5f-43bb-8139-ca10ce0092e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:05.043590 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d6a97-9a5f-43bb-8139-ca10ce0092e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:05.043892 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d6a97-9a5f-43bb-8139-ca10ce0092e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:05.044146 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d6a97-9a5f-43bb-8139-ca10ce0092e1 admin admin] PATH_INFO: `/v3/users/8742a73e9b4f46deb0b5c9e1756ffe2e` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:05.045674 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d6a97-9a5f-43bb-8139-ca10ce0092e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8742a73e9b4f46deb0b5c9e1756ffe2e)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:05.047752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d6a97-9a5f-43bb-8139-ca10ce0092e1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:05.102005 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-050d6a97-9a5f-43bb-8139-ca10ce0092e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8742a73e9b4f46deb0b5c9e1756ffe2e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:05.102697 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-050d6a97-9a5f-43bb-8139-ca10ce0092e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8742a73e9b4f46deb0b5c9e1756ffe2e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:05.103381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-050d6a97-9a5f-43bb-8139-ca10ce0092e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8742a73e9b4f46deb0b5c9e1756ffe2e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:05.113144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-050d6a97-9a5f-43bb-8139-ca10ce0092e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8742a73e9b4f46deb0b5c9e1756ffe2e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:05.123909 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-050d6a97-9a5f-43bb-8139-ca10ce0092e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8742a73e9b4f46deb0b5c9e1756ffe2e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:05.142049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1306/2596] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:07:05 2022] DELETE /identity/v3/users/8742a73e9b4f46deb0b5c9e1756ffe2e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:07:05.323465 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f86a1f-4caa-4530-86a5-0ecd19b69549 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:05.334925 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f86a1f-4caa-4530-86a5-0ecd19b69549 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:05.399724 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae99de-e9c3-462e-8804-ac684618abff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:05.400520 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae99de-e9c3-462e-8804-ac684618abff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:05.400798 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae99de-e9c3-462e-8804-ac684618abff admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:05.401071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae99de-e9c3-462e-8804-ac684618abff admin admin] PATH_INFO: `/v3/projects/d865cf7b3e5c4291a0204f396143083a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:05.402258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae99de-e9c3-462e-8804-ac684618abff admin admin] RBAC: Authorizing `identity:delete_project(project_id=d865cf7b3e5c4291a0204f396143083a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:05.405227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae99de-e9c3-462e-8804-ac684618abff admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:05.453752 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f2ae99de-e9c3-462e-8804-ac684618abff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd865cf7b3e5c4291a0204f396143083a', 'request_id': 'req-f2ae99de-e9c3-462e-8804-ac684618abff'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:05.461747 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f2ae99de-e9c3-462e-8804-ac684618abff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd865cf7b3e5c4291a0204f396143083a', 'request_id': 'req-f2ae99de-e9c3-462e-8804-ac684618abff'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:05.468554 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f2ae99de-e9c3-462e-8804-ac684618abff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd865cf7b3e5c4291a0204f396143083a', 'request_id': 'req-f2ae99de-e9c3-462e-8804-ac684618abff'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:05.478711 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f2ae99de-e9c3-462e-8804-ac684618abff admin admin] The token cache is being invalidate because project d865cf7b3e5c4291a0204f396143083a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:07:05.479027 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f2ae99de-e9c3-462e-8804-ac684618abff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:05.479660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f2ae99de-e9c3-462e-8804-ac684618abff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:05.481183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1291/2597] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:07:05 2022] DELETE /identity/v3/projects/d865cf7b3e5c4291a0204f396143083a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:05.487055 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ac574-de42-4080-8149-42f22addaecf None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:05.487373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ac574-de42-4080-8149-42f22addaecf None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:05.487683 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ac574-de42-4080-8149-42f22addaecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:05.505823 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-404ac574-de42-4080-8149-42f22addaecf None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:05.507079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-404ac574-de42-4080-8149-42f22addaecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:05.567964 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1307/2598] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:05.576248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b11c99-7397-44fb-8ce7-ee7f765a76e5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:05.576561 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b11c99-7397-44fb-8ce7-ee7f765a76e5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:05.576835 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b11c99-7397-44fb-8ce7-ee7f765a76e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:05.595369 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f4b11c99-7397-44fb-8ce7-ee7f765a76e5 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:05.596549 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f4b11c99-7397-44fb-8ce7-ee7f765a76e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:05.604338 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1292/2599] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:05.609324 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ac574-de42-4080-8149-42f22addaecf None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:05.619300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ac574-de42-4080-8149-42f22addaecf None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:05.624206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c571d-7c4a-4265-84fe-04ab4cad641c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:05.624864 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c571d-7c4a-4265-84fe-04ab4cad641c admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:05.625150 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c571d-7c4a-4265-84fe-04ab4cad641c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:05.625429 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c571d-7c4a-4265-84fe-04ab4cad641c admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:05.626188 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c571d-7c4a-4265-84fe-04ab4cad641c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:07:05.626722 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c571d-7c4a-4265-84fe-04ab4cad641c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:05.628269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c571d-7c4a-4265-84fe-04ab4cad641c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:05.641001 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1308/2600] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:07:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:07:05.645185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b11c99-7397-44fb-8ce7-ee7f765a76e5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:05.651243 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:05.655552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b11c99-7397-44fb-8ce7-ee7f765a76e5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:05.661969 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad99442c-7b43-4d53-b29d-18f33eec200b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:05.662674 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99442c-7b43-4d53-b29d-18f33eec200b admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:05.662970 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99442c-7b43-4d53-b29d-18f33eec200b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:05.663405 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99442c-7b43-4d53-b29d-18f33eec200b admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:05.664156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad99442c-7b43-4d53-b29d-18f33eec200b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:05.666199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad99442c-7b43-4d53-b29d-18f33eec200b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:05.692867 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1293/2601] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:07:05 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:07:05.696697 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c571d-7c4a-4265-84fe-04ab4cad641c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:05.707399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c571d-7c4a-4265-84fe-04ab4cad641c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:05.716383 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:05.766665 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beced6b-f151-4e5d-8530-cbf2959ed25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:05.767340 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beced6b-f151-4e5d-8530-cbf2959ed25b admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:05.767624 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beced6b-f151-4e5d-8530-cbf2959ed25b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:05.767942 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beced6b-f151-4e5d-8530-cbf2959ed25b admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:05.768623 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beced6b-f151-4e5d-8530-cbf2959ed25b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:05.770524 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beced6b-f151-4e5d-8530-cbf2959ed25b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:05.790189 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1309/2602] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:07:05 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:07:05.794228 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad99442c-7b43-4d53-b29d-18f33eec200b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:05.804728 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad99442c-7b43-4d53-b29d-18f33eec200b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:05.809245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7744d-51a2-41d3-95b8-c2c52e49e257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:05.809895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7744d-51a2-41d3-95b8-c2c52e49e257 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:05.810177 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7744d-51a2-41d3-95b8-c2c52e49e257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:05.810525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7744d-51a2-41d3-95b8-c2c52e49e257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:05.811189 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7744d-51a2-41d3-95b8-c2c52e49e257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:05.812939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7744d-51a2-41d3-95b8-c2c52e49e257 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:05.824210 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1294/2603] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:05.828796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beced6b-f151-4e5d-8530-cbf2959ed25b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:05.839020 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beced6b-f151-4e5d-8530-cbf2959ed25b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:05.844454 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191982f1-e849-4c8f-b78a-40105a5d3d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:05.845143 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191982f1-e849-4c8f-b78a-40105a5d3d68 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:05.845452 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191982f1-e849-4c8f-b78a-40105a5d3d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:05.845733 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191982f1-e849-4c8f-b78a-40105a5d3d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:05.846462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191982f1-e849-4c8f-b78a-40105a5d3d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:05.848092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191982f1-e849-4c8f-b78a-40105a5d3d68 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:05.858854 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1310/2604] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:05.863022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7744d-51a2-41d3-95b8-c2c52e49e257 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:05.873211 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7744d-51a2-41d3-95b8-c2c52e49e257 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:05.878383 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12effad-19db-41d0-a69d-877df2c49e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:05.879422 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12effad-19db-41d0-a69d-877df2c49e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:05.879831 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12effad-19db-41d0-a69d-877df2c49e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:05.880200 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12effad-19db-41d0-a69d-877df2c49e68 admin admin] PATH_INFO: `/v3/projects/53528b6015f2423db9e79f9090643158/users/fd402a8feac24f43b1bb7a64fcca1683/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:05.895802 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12effad-19db-41d0-a69d-877df2c49e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53528b6015f2423db9e79f9090643158, user_id=fd402a8feac24f43b1bb7a64fcca1683, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:05.898475 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12effad-19db-41d0-a69d-877df2c49e68 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:05.912594 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1295/2605] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:07:05 2022] PUT /identity/v3/projects/53528b6015f2423db9e79f9090643158/users/fd402a8feac24f43b1bb7a64fcca1683/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:05.923357 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42513a-4bcb-40cc-8827-0164f0cef2e7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:05.923691 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42513a-4bcb-40cc-8827-0164f0cef2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:05.923979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42513a-4bcb-40cc-8827-0164f0cef2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:05.952061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-cc42513a-4bcb-40cc-8827-0164f0cef2e7 None None] MFA Rules not processed for user `fd402a8feac24f43b1bb7a64fcca1683`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:05.953393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cc42513a-4bcb-40cc-8827-0164f0cef2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:06.021337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1311/2606] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:06.032743 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12effad-19db-41d0-a69d-877df2c49e68 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:06.034467 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e12effad-19db-41d0-a69d-877df2c49e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:06.107432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12effad-19db-41d0-a69d-877df2c49e68 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:06.112918 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd46e6-be53-4e45-ad40-9b4437eda749 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:06.113552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd46e6-be53-4e45-ad40-9b4437eda749 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:06.113871 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd46e6-be53-4e45-ad40-9b4437eda749 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:06.114147 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd46e6-be53-4e45-ad40-9b4437eda749 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:06.122363 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd46e6-be53-4e45-ad40-9b4437eda749 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:06.124946 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd46e6-be53-4e45-ad40-9b4437eda749 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:06.133836 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1296/2607] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:13.421295 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42513a-4bcb-40cc-8827-0164f0cef2e7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:13.490677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42513a-4bcb-40cc-8827-0164f0cef2e7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:13.495135 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3a3de-9913-4484-87ef-b490b77db3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:13.496061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3a3de-9913-4484-87ef-b490b77db3e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:13.496376 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3a3de-9913-4484-87ef-b490b77db3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:13.496654 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3a3de-9913-4484-87ef-b490b77db3e2 admin admin] PATH_INFO: `/v3/users/fd402a8feac24f43b1bb7a64fcca1683` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:13.497908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3a3de-9913-4484-87ef-b490b77db3e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd402a8feac24f43b1bb7a64fcca1683)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:13.499886 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3a3de-9913-4484-87ef-b490b77db3e2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:13.522519 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:13.551148 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a8f3a3de-9913-4484-87ef-b490b77db3e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd402a8feac24f43b1bb7a64fcca1683'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:13.551810 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a8f3a3de-9913-4484-87ef-b490b77db3e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd402a8feac24f43b1bb7a64fcca1683'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:13.552417 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a8f3a3de-9913-4484-87ef-b490b77db3e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd402a8feac24f43b1bb7a64fcca1683'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:13.560543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a8f3a3de-9913-4484-87ef-b490b77db3e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd402a8feac24f43b1bb7a64fcca1683'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:13.567763 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a8f3a3de-9913-4484-87ef-b490b77db3e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd402a8feac24f43b1bb7a64fcca1683'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:13.585057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1312/2608] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:07:13 2022] DELETE /identity/v3/users/fd402a8feac24f43b1bb7a64fcca1683 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:13.589735 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f7dce-2537-46d6-abf9-35cc7cfc5718 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:13.590219 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f7dce-2537-46d6-abf9-35cc7cfc5718 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:13.590323 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f7dce-2537-46d6-abf9-35cc7cfc5718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:13.608618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-045f7dce-2537-46d6-abf9-35cc7cfc5718 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:13.609807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-045f7dce-2537-46d6-abf9-35cc7cfc5718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:13.671856 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1297/2609] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:13.682531 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3a3de-9913-4484-87ef-b490b77db3e2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:13.683952 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a8f3a3de-9913-4484-87ef-b490b77db3e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:13.746531 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3a3de-9913-4484-87ef-b490b77db3e2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:13.750348 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b25ba-e890-495f-ab57-c069ec00da2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:13.751032 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b25ba-e890-495f-ab57-c069ec00da2d service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:13.751315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b25ba-e890-495f-ab57-c069ec00da2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:13.751589 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b25ba-e890-495f-ab57-c069ec00da2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:13.760739 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b25ba-e890-495f-ab57-c069ec00da2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:13.762214 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b25ba-e890-495f-ab57-c069ec00da2d service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:13.771989 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1313/2610] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:14.054646 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f7dce-2537-46d6-abf9-35cc7cfc5718 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:14.056092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-045f7dce-2537-46d6-abf9-35cc7cfc5718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:14.069347 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f7dce-2537-46d6-abf9-35cc7cfc5718 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:14.074131 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980cf2b-57c4-470b-9595-935257ede9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:14.075039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980cf2b-57c4-470b-9595-935257ede9c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:14.075935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980cf2b-57c4-470b-9595-935257ede9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:14.076373 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980cf2b-57c4-470b-9595-935257ede9c4 admin admin] PATH_INFO: `/v3/projects/53528b6015f2423db9e79f9090643158` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:14.078934 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980cf2b-57c4-470b-9595-935257ede9c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53528b6015f2423db9e79f9090643158)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:14.081689 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980cf2b-57c4-470b-9595-935257ede9c4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:14.114874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:14.131415 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6980cf2b-57c4-470b-9595-935257ede9c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53528b6015f2423db9e79f9090643158', 'request_id': 'req-6980cf2b-57c4-470b-9595-935257ede9c4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:14.139317 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6980cf2b-57c4-470b-9595-935257ede9c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53528b6015f2423db9e79f9090643158', 'request_id': 'req-6980cf2b-57c4-470b-9595-935257ede9c4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:14.145550 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6980cf2b-57c4-470b-9595-935257ede9c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53528b6015f2423db9e79f9090643158', 'request_id': 'req-6980cf2b-57c4-470b-9595-935257ede9c4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:14.156956 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6980cf2b-57c4-470b-9595-935257ede9c4 admin admin] The token cache is being invalidate because project 53528b6015f2423db9e79f9090643158 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:07:14.157236 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6980cf2b-57c4-470b-9595-935257ede9c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:14.157872 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6980cf2b-57c4-470b-9595-935257ede9c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:14.159452 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1298/2611] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:07:14 2022] DELETE /identity/v3/projects/53528b6015f2423db9e79f9090643158 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:14.164535 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58261618-4a6d-4d8e-895c-60cda0113bd9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:14.164855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58261618-4a6d-4d8e-895c-60cda0113bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:14.165136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58261618-4a6d-4d8e-895c-60cda0113bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:14.182661 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-58261618-4a6d-4d8e-895c-60cda0113bd9 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:14.183860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-58261618-4a6d-4d8e-895c-60cda0113bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:14.243183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1314/2612] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:14.251703 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8527870-ff05-4b04-a7d8-bbc27b50af12 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:14.251921 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8527870-ff05-4b04-a7d8-bbc27b50af12 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:14.252195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8527870-ff05-4b04-a7d8-bbc27b50af12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:14.271255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e8527870-ff05-4b04-a7d8-bbc27b50af12 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:14.272470 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e8527870-ff05-4b04-a7d8-bbc27b50af12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:14.280928 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1299/2613] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:14.285217 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58261618-4a6d-4d8e-895c-60cda0113bd9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:14.295352 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58261618-4a6d-4d8e-895c-60cda0113bd9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:14.299972 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47566d10-0785-4659-8909-1fc6927edfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:14.300622 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47566d10-0785-4659-8909-1fc6927edfeb admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:14.300904 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47566d10-0785-4659-8909-1fc6927edfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:14.301179 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47566d10-0785-4659-8909-1fc6927edfeb admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:14.301906 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47566d10-0785-4659-8909-1fc6927edfeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:07:14.302211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47566d10-0785-4659-8909-1fc6927edfeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:14.303912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47566d10-0785-4659-8909-1fc6927edfeb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:14.316792 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1315/2614] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:07:14.321171 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8527870-ff05-4b04-a7d8-bbc27b50af12 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:14.331469 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8527870-ff05-4b04-a7d8-bbc27b50af12 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:14.336402 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86672630-ba85-47b9-9760-b2d8b828bd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:14.337061 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86672630-ba85-47b9-9760-b2d8b828bd65 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:14.337342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86672630-ba85-47b9-9760-b2d8b828bd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:14.337615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86672630-ba85-47b9-9760-b2d8b828bd65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:14.338300 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86672630-ba85-47b9-9760-b2d8b828bd65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:14.340459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86672630-ba85-47b9-9760-b2d8b828bd65 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:14.368176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1300/2615] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:07:14 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:07:14.373649 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47566d10-0785-4659-8909-1fc6927edfeb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:14.383657 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47566d10-0785-4659-8909-1fc6927edfeb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:14.413180 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:14.442550 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbaf76-8be3-46bb-9e45-c663e8e39f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:14.443252 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbaf76-8be3-46bb-9e45-c663e8e39f23 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:14.443540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbaf76-8be3-46bb-9e45-c663e8e39f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:14.443821 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbaf76-8be3-46bb-9e45-c663e8e39f23 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:14.444508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbaf76-8be3-46bb-9e45-c663e8e39f23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:14.446470 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbaf76-8be3-46bb-9e45-c663e8e39f23 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:14.466057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1316/2616] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:07:14 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:07:14.470763 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86672630-ba85-47b9-9760-b2d8b828bd65 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:14.481030 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86672630-ba85-47b9-9760-b2d8b828bd65 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:14.485922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854660f-2267-4eba-9bf2-cb92bcc4eb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:14.486598 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854660f-2267-4eba-9bf2-cb92bcc4eb1f admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:14.486895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854660f-2267-4eba-9bf2-cb92bcc4eb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:14.487170 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854660f-2267-4eba-9bf2-cb92bcc4eb1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:14.487892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854660f-2267-4eba-9bf2-cb92bcc4eb1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:14.489533 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854660f-2267-4eba-9bf2-cb92bcc4eb1f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:14.501097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1301/2617] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:14.505133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbaf76-8be3-46bb-9e45-c663e8e39f23 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:14.515706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbaf76-8be3-46bb-9e45-c663e8e39f23 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:14.520645 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8a89a-d23d-4442-aa44-2143fb1b4105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:14.521343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8a89a-d23d-4442-aa44-2143fb1b4105 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:14.521632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8a89a-d23d-4442-aa44-2143fb1b4105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:14.521939 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8a89a-d23d-4442-aa44-2143fb1b4105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:14.522715 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8a89a-d23d-4442-aa44-2143fb1b4105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:14.524576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8a89a-d23d-4442-aa44-2143fb1b4105 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:14.534679 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1317/2618] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:14.538685 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854660f-2267-4eba-9bf2-cb92bcc4eb1f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:14.549321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854660f-2267-4eba-9bf2-cb92bcc4eb1f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:14.554380 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51553385-0e6d-41cd-88ea-aef689df4662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:14.555319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51553385-0e6d-41cd-88ea-aef689df4662 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:14.555591 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51553385-0e6d-41cd-88ea-aef689df4662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:14.555870 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51553385-0e6d-41cd-88ea-aef689df4662 admin admin] PATH_INFO: `/v3/projects/ddd0d6cc6ac0450fb85d4141fb62f4d5/users/7b6aeacd0c574098827be19ad11534fc/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:14.572327 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51553385-0e6d-41cd-88ea-aef689df4662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddd0d6cc6ac0450fb85d4141fb62f4d5, user_id=7b6aeacd0c574098827be19ad11534fc, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:14.575320 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51553385-0e6d-41cd-88ea-aef689df4662 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:14.588906 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1302/2619] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:07:14 2022] PUT /identity/v3/projects/ddd0d6cc6ac0450fb85d4141fb62f4d5/users/7b6aeacd0c574098827be19ad11534fc/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:07:14.599205 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd2e5a-b429-4115-9053-ead3e3b3671a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:14.599501 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd2e5a-b429-4115-9053-ead3e3b3671a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:14.599787 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd2e5a-b429-4115-9053-ead3e3b3671a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:14.630243 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1dcd2e5a-b429-4115-9053-ead3e3b3671a None None] MFA Rules not processed for user `7b6aeacd0c574098827be19ad11534fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:14.631618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1dcd2e5a-b429-4115-9053-ead3e3b3671a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:14.657015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51553385-0e6d-41cd-88ea-aef689df4662 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:14.659156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-51553385-0e6d-41cd-88ea-aef689df4662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:14.703547 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1318/2620] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:14.715208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd2e5a-b429-4115-9053-ead3e3b3671a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:14.716830 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1dcd2e5a-b429-4115-9053-ead3e3b3671a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:14.731376 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51553385-0e6d-41cd-88ea-aef689df4662 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:14.737541 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83443404-2716-408e-a5b6-c61126f1008f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '880d0299c38c4eb897c669a512e599be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:14.738371 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83443404-2716-408e-a5b6-c61126f1008f service placement] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:14.738693 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83443404-2716-408e-a5b6-c61126f1008f service placement] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:14.738980 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83443404-2716-408e-a5b6-c61126f1008f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:14.740867 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-83443404-2716-408e-a5b6-c61126f1008f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:14.753932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83443404-2716-408e-a5b6-c61126f1008f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:14.755687 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83443404-2716-408e-a5b6-c61126f1008f service placement] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:14.769145 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1303/2621] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:07:14.799311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd2e5a-b429-4115-9053-ead3e3b3671a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:14.804869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe1c55-bfbd-48f4-b245-0359b0f33822 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:14.805522 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe1c55-bfbd-48f4-b245-0359b0f33822 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:14.805814 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe1c55-bfbd-48f4-b245-0359b0f33822 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:14.806092 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe1c55-bfbd-48f4-b245-0359b0f33822 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:14.816535 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe1c55-bfbd-48f4-b245-0359b0f33822 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:14.818191 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe1c55-bfbd-48f4-b245-0359b0f33822 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:14.829064 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1319/2622] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:07:15.055118 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83443404-2716-408e-a5b6-c61126f1008f service placement] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:15.116325 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:15.133785 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83443404-2716-408e-a5b6-c61126f1008f service placement] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:15.138983 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a6b10-6c06-4b2e-88ab-394ae9c95239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:15.139849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a6b10-6c06-4b2e-88ab-394ae9c95239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:15.140130 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a6b10-6c06-4b2e-88ab-394ae9c95239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:15.140404 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a6b10-6c06-4b2e-88ab-394ae9c95239 admin admin] PATH_INFO: `/v3/users/7b6aeacd0c574098827be19ad11534fc` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:15.141607 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a6b10-6c06-4b2e-88ab-394ae9c95239 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b6aeacd0c574098827be19ad11534fc)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:15.143571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a6b10-6c06-4b2e-88ab-394ae9c95239 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:15.190515 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8e6a6b10-6c06-4b2e-88ab-394ae9c95239 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b6aeacd0c574098827be19ad11534fc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:15.191195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8e6a6b10-6c06-4b2e-88ab-394ae9c95239 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b6aeacd0c574098827be19ad11534fc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:15.198008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8e6a6b10-6c06-4b2e-88ab-394ae9c95239 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b6aeacd0c574098827be19ad11534fc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:15.210202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8e6a6b10-6c06-4b2e-88ab-394ae9c95239 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b6aeacd0c574098827be19ad11534fc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:15.210869 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8e6a6b10-6c06-4b2e-88ab-394ae9c95239 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b6aeacd0c574098827be19ad11534fc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:15.224776 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1304/2623] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:07:15 2022] DELETE /identity/v3/users/7b6aeacd0c574098827be19ad11534fc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:15.229439 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2cd31-9f63-4835-b57e-8ccbc3a4c1e8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:15.229744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2cd31-9f63-4835-b57e-8ccbc3a4c1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:15.230020 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2cd31-9f63-4835-b57e-8ccbc3a4c1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:15.252211 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1ed2cd31-9f63-4835-b57e-8ccbc3a4c1e8 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:15.253833 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1ed2cd31-9f63-4835-b57e-8ccbc3a4c1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:15.274122 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:15.318896 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1320/2624] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:15.328953 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a6b10-6c06-4b2e-88ab-394ae9c95239 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:15.330391 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8e6a6b10-6c06-4b2e-88ab-394ae9c95239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:15.394317 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a6b10-6c06-4b2e-88ab-394ae9c95239 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:15.398211 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7058a-aa57-40b3-840b-9e400489b747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:15.399275 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7058a-aa57-40b3-840b-9e400489b747 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:15.399606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7058a-aa57-40b3-840b-9e400489b747 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:15.399937 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7058a-aa57-40b3-840b-9e400489b747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:15.411286 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7058a-aa57-40b3-840b-9e400489b747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:15.412743 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7058a-aa57-40b3-840b-9e400489b747 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:15.424111 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1305/2625] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:15.713986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2cd31-9f63-4835-b57e-8ccbc3a4c1e8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:15.716693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1ed2cd31-9f63-4835-b57e-8ccbc3a4c1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:15.750681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2cd31-9f63-4835-b57e-8ccbc3a4c1e8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:15.763416 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862a295-3113-4661-a0f1-1a523c878d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:15.764459 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862a295-3113-4661-a0f1-1a523c878d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:15.764741 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862a295-3113-4661-a0f1-1a523c878d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:15.765018 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862a295-3113-4661-a0f1-1a523c878d87 admin admin] PATH_INFO: `/v3/projects/ddd0d6cc6ac0450fb85d4141fb62f4d5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:15.766231 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862a295-3113-4661-a0f1-1a523c878d87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddd0d6cc6ac0450fb85d4141fb62f4d5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:15.768178 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862a295-3113-4661-a0f1-1a523c878d87 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:15.821681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5862a295-3113-4661-a0f1-1a523c878d87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddd0d6cc6ac0450fb85d4141fb62f4d5', 'request_id': 'req-5862a295-3113-4661-a0f1-1a523c878d87'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:15.831972 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5862a295-3113-4661-a0f1-1a523c878d87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddd0d6cc6ac0450fb85d4141fb62f4d5', 'request_id': 'req-5862a295-3113-4661-a0f1-1a523c878d87'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:15.838674 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5862a295-3113-4661-a0f1-1a523c878d87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddd0d6cc6ac0450fb85d4141fb62f4d5', 'request_id': 'req-5862a295-3113-4661-a0f1-1a523c878d87'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:15.850451 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5862a295-3113-4661-a0f1-1a523c878d87 admin admin] The token cache is being invalidate because project ddd0d6cc6ac0450fb85d4141fb62f4d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:07:15.850932 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5862a295-3113-4661-a0f1-1a523c878d87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:15.851725 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5862a295-3113-4661-a0f1-1a523c878d87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:15.853516 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1321/2626] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:07:15 2022] DELETE /identity/v3/projects/ddd0d6cc6ac0450fb85d4141fb62f4d5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:15.859434 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89731a0a-749b-43a9-8e42-a54dfc8594a6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:15.859947 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89731a0a-749b-43a9-8e42-a54dfc8594a6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:15.860058 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89731a0a-749b-43a9-8e42-a54dfc8594a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:15.878210 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-89731a0a-749b-43a9-8e42-a54dfc8594a6 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:15.879451 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-89731a0a-749b-43a9-8e42-a54dfc8594a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:15.941524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1306/2627] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:15.949213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b2509-7e33-4690-bb32-8c7b25c22074 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:15.949504 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b2509-7e33-4690-bb32-8c7b25c22074 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:15.949785 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b2509-7e33-4690-bb32-8c7b25c22074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:15.968946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5a2b2509-7e33-4690-bb32-8c7b25c22074 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:15.970142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5a2b2509-7e33-4690-bb32-8c7b25c22074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:15.977964 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1322/2628] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:15.983183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89731a0a-749b-43a9-8e42-a54dfc8594a6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:15.989829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:15.993569 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89731a0a-749b-43a9-8e42-a54dfc8594a6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:15.999803 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2300f-2bc2-431e-a6d0-4e089eaa5132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:16.000632 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2300f-2bc2-431e-a6d0-4e089eaa5132 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:16.001104 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2300f-2bc2-431e-a6d0-4e089eaa5132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:16.001572 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2300f-2bc2-431e-a6d0-4e089eaa5132 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:16.002301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2300f-2bc2-431e-a6d0-4e089eaa5132 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:07:16.002641 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2300f-2bc2-431e-a6d0-4e089eaa5132 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:16.004244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2300f-2bc2-431e-a6d0-4e089eaa5132 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:16.016052 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1307/2629] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:07:16.020285 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b2509-7e33-4690-bb32-8c7b25c22074 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:16.030545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b2509-7e33-4690-bb32-8c7b25c22074 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:16.035302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd93b21-bdf8-444a-806a-762a5a58d656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:16.035953 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd93b21-bdf8-444a-806a-762a5a58d656 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:16.036240 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd93b21-bdf8-444a-806a-762a5a58d656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:16.036516 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd93b21-bdf8-444a-806a-762a5a58d656 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:16.037201 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd93b21-bdf8-444a-806a-762a5a58d656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:16.039131 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd93b21-bdf8-444a-806a-762a5a58d656 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:16.065438 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1323/2630] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:07:16 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:07:16.069613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2300f-2bc2-431e-a6d0-4e089eaa5132 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:16.079813 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2300f-2bc2-431e-a6d0-4e089eaa5132 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:16.137253 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357757a-88ce-419c-9b69-e92e8bdf610b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:16.138496 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357757a-88ce-419c-9b69-e92e8bdf610b admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:16.138901 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357757a-88ce-419c-9b69-e92e8bdf610b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:16.139117 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357757a-88ce-419c-9b69-e92e8bdf610b admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:16.139840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357757a-88ce-419c-9b69-e92e8bdf610b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:16.141676 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357757a-88ce-419c-9b69-e92e8bdf610b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:16.161345 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1308/2631] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:07:16 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:07:16.165490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd93b21-bdf8-444a-806a-762a5a58d656 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:16.175936 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd93b21-bdf8-444a-806a-762a5a58d656 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:16.180558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c96ad8-55a1-474a-ac21-c299af3c2570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:16.181221 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c96ad8-55a1-474a-ac21-c299af3c2570 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:16.181499 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c96ad8-55a1-474a-ac21-c299af3c2570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:16.181772 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c96ad8-55a1-474a-ac21-c299af3c2570 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:16.182485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c96ad8-55a1-474a-ac21-c299af3c2570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:16.184088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c96ad8-55a1-474a-ac21-c299af3c2570 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:16.195975 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1324/2632] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:16.200007 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357757a-88ce-419c-9b69-e92e8bdf610b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:16.211079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357757a-88ce-419c-9b69-e92e8bdf610b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:16.215922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3428b2-0783-4e68-86df-c0508ea27fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:16.217607 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3428b2-0783-4e68-86df-c0508ea27fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:16.217930 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3428b2-0783-4e68-86df-c0508ea27fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:16.218246 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3428b2-0783-4e68-86df-c0508ea27fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:16.219053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3428b2-0783-4e68-86df-c0508ea27fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:16.220671 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3428b2-0783-4e68-86df-c0508ea27fc8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:16.230526 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1309/2633] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:16.234846 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c96ad8-55a1-474a-ac21-c299af3c2570 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:16.244699 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c96ad8-55a1-474a-ac21-c299af3c2570 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:16.249616 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8e5fe-512e-49ef-8a1c-1e07e2656c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:16.254251 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8e5fe-512e-49ef-8a1c-1e07e2656c3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:16.254543 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8e5fe-512e-49ef-8a1c-1e07e2656c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:16.254855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8e5fe-512e-49ef-8a1c-1e07e2656c3c admin admin] PATH_INFO: `/v3/projects/afdf4f1e52dd4a17b939a062ddeb9b0a/users/ab1c974a5ec645a7a6effa2a371a3b74/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:16.271760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8e5fe-512e-49ef-8a1c-1e07e2656c3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=afdf4f1e52dd4a17b939a062ddeb9b0a, user_id=ab1c974a5ec645a7a6effa2a371a3b74, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:16.274693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8e5fe-512e-49ef-8a1c-1e07e2656c3c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:16.288905 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1325/2634] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:07:16 2022] PUT /identity/v3/projects/afdf4f1e52dd4a17b939a062ddeb9b0a/users/ab1c974a5ec645a7a6effa2a371a3b74/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:16.303487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44298c8-1a0f-4726-b973-8f232b11cbd7 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:16.303780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44298c8-1a0f-4726-b973-8f232b11cbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:16.304062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44298c8-1a0f-4726-b973-8f232b11cbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:16.331582 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-a44298c8-1a0f-4726-b973-8f232b11cbd7 None None] MFA Rules not processed for user `ab1c974a5ec645a7a6effa2a371a3b74`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:16.332717 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a44298c8-1a0f-4726-b973-8f232b11cbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:16.407054 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1310/2635] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:16.411485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8e5fe-512e-49ef-8a1c-1e07e2656c3c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:16.468658 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:16.488295 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8e5fe-512e-49ef-8a1c-1e07e2656c3c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:16.493756 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf8757-7fe7-4c37-8df2-78aa9ecc3c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:16.494566 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf8757-7fe7-4c37-8df2-78aa9ecc3c85 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:16.495118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf8757-7fe7-4c37-8df2-78aa9ecc3c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:16.495397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf8757-7fe7-4c37-8df2-78aa9ecc3c85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:16.496090 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf8757-7fe7-4c37-8df2-78aa9ecc3c85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:16.498646 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf8757-7fe7-4c37-8df2-78aa9ecc3c85 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:16.531542 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1326/2636] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:07:16 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:07:16.532809 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44298c8-1a0f-4726-b973-8f232b11cbd7 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:16.544398 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44298c8-1a0f-4726-b973-8f232b11cbd7 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:16.604748 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ad8ca-4e5f-4bcf-98ef-2e6a5e7b2840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:16.605333 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ad8ca-4e5f-4bcf-98ef-2e6a5e7b2840 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:16.605619 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ad8ca-4e5f-4bcf-98ef-2e6a5e7b2840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:16.605897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ad8ca-4e5f-4bcf-98ef-2e6a5e7b2840 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:16.606826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ad8ca-4e5f-4bcf-98ef-2e6a5e7b2840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:16.608751 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ad8ca-4e5f-4bcf-98ef-2e6a5e7b2840 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:16.615409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:16.634027 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1311/2637] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:07:16 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:07:16.635267 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf8757-7fe7-4c37-8df2-78aa9ecc3c85 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:16.646895 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf8757-7fe7-4c37-8df2-78aa9ecc3c85 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:16.651902 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944124a1-3620-4f3f-b37d-28b006905730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:16.652571 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944124a1-3620-4f3f-b37d-28b006905730 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:16.652861 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944124a1-3620-4f3f-b37d-28b006905730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:16.653144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944124a1-3620-4f3f-b37d-28b006905730 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:16.654731 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944124a1-3620-4f3f-b37d-28b006905730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:16.656266 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944124a1-3620-4f3f-b37d-28b006905730 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:16.667970 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1327/2638] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:16.672238 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ad8ca-4e5f-4bcf-98ef-2e6a5e7b2840 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:16.682291 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ad8ca-4e5f-4bcf-98ef-2e6a5e7b2840 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:16.687837 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ce606-8d3a-4eeb-953b-8b3f8a413e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:16.688777 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ce606-8d3a-4eeb-953b-8b3f8a413e82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:16.689059 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ce606-8d3a-4eeb-953b-8b3f8a413e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:16.689333 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ce606-8d3a-4eeb-953b-8b3f8a413e82 admin admin] PATH_INFO: `/v3/projects/9370a62ad2bb4e7c9346b8e0ae422e16/users/e024a74ff44941e6be067b6c1dbdadbe/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:16.706529 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ce606-8d3a-4eeb-953b-8b3f8a413e82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9370a62ad2bb4e7c9346b8e0ae422e16, user_id=e024a74ff44941e6be067b6c1dbdadbe, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:16.709957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ce606-8d3a-4eeb-953b-8b3f8a413e82 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:16.728284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1312/2639] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:07:16 2022] PUT /identity/v3/projects/9370a62ad2bb4e7c9346b8e0ae422e16/users/e024a74ff44941e6be067b6c1dbdadbe/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:16.730829 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944124a1-3620-4f3f-b37d-28b006905730 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:16.746361 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944124a1-3620-4f3f-b37d-28b006905730 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:16.803710 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60282f-46de-49f2-94d8-67af65c2cab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:16.804384 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60282f-46de-49f2-94d8-67af65c2cab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:16.804668 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60282f-46de-49f2-94d8-67af65c2cab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:16.804942 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60282f-46de-49f2-94d8-67af65c2cab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:16.805665 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60282f-46de-49f2-94d8-67af65c2cab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:16.807286 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60282f-46de-49f2-94d8-67af65c2cab0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:16.817868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1328/2640] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:16.822917 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ce606-8d3a-4eeb-953b-8b3f8a413e82 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:16.834964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ce606-8d3a-4eeb-953b-8b3f8a413e82 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:16.840003 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6473c89-d227-483a-a4ae-a71037f39679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:16.841614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6473c89-d227-483a-a4ae-a71037f39679 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:16.841896 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6473c89-d227-483a-a4ae-a71037f39679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:16.842172 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6473c89-d227-483a-a4ae-a71037f39679 admin admin] PATH_INFO: `/v3/projects/9370a62ad2bb4e7c9346b8e0ae422e16/users/e024a74ff44941e6be067b6c1dbdadbe/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:16.844253 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6473c89-d227-483a-a4ae-a71037f39679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9370a62ad2bb4e7c9346b8e0ae422e16, user_id=e024a74ff44941e6be067b6c1dbdadbe, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:16.846930 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6473c89-d227-483a-a4ae-a71037f39679 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:16.858116 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1313/2641] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:07:16 2022] PUT /identity/v3/projects/9370a62ad2bb4e7c9346b8e0ae422e16/users/e024a74ff44941e6be067b6c1dbdadbe/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:07:16.869876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517913cc-446f-49fa-b22f-300f1d9154b2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:16.870172 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517913cc-446f-49fa-b22f-300f1d9154b2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:16.871132 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517913cc-446f-49fa-b22f-300f1d9154b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:16.899085 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-517913cc-446f-49fa-b22f-300f1d9154b2 None None] MFA Rules not processed for user `e024a74ff44941e6be067b6c1dbdadbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:16.900415 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-517913cc-446f-49fa-b22f-300f1d9154b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:16.979365 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1329/2642] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:07:16.992001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6473c89-d227-483a-a4ae-a71037f39679 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:16.993983 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a6473c89-d227-483a-a4ae-a71037f39679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:17.055298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6473c89-d227-483a-a4ae-a71037f39679 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:17.059949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81760dd-aeb3-4025-855c-5a7eb6f06cf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:17.060801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81760dd-aeb3-4025-855c-5a7eb6f06cf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:17.061100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81760dd-aeb3-4025-855c-5a7eb6f06cf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:17.061381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81760dd-aeb3-4025-855c-5a7eb6f06cf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:17.071309 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81760dd-aeb3-4025-855c-5a7eb6f06cf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:17.073492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81760dd-aeb3-4025-855c-5a7eb6f06cf1 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:17.084342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1314/2643] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:17.159516 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517913cc-446f-49fa-b22f-300f1d9154b2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:17.169675 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517913cc-446f-49fa-b22f-300f1d9154b2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:17.173347 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f06fe7-817f-431a-889b-3071bd03737f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:17.173992 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f06fe7-817f-431a-889b-3071bd03737f service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:17.174276 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f06fe7-817f-431a-889b-3071bd03737f service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:17.174562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f06fe7-817f-431a-889b-3071bd03737f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:17.214132 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:17.232416 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f06fe7-817f-431a-889b-3071bd03737f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:17.233797 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f06fe7-817f-431a-889b-3071bd03737f service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:17.242833 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1330/2644] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:26.110089 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81760dd-aeb3-4025-855c-5a7eb6f06cf1 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:26.175911 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81760dd-aeb3-4025-855c-5a7eb6f06cf1 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:26.180638 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1c60c-0387-4320-8e3e-6361749c4411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:26.181524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1c60c-0387-4320-8e3e-6361749c4411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:26.181810 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1c60c-0387-4320-8e3e-6361749c4411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:26.182090 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1c60c-0387-4320-8e3e-6361749c4411 admin admin] PATH_INFO: `/v3/users/ab1c974a5ec645a7a6effa2a371a3b74` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:26.183333 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c1c60c-0387-4320-8e3e-6361749c4411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab1c974a5ec645a7a6effa2a371a3b74)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:26.185292 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c1c60c-0387-4320-8e3e-6361749c4411 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:26.245730 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-85c1c60c-0387-4320-8e3e-6361749c4411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab1c974a5ec645a7a6effa2a371a3b74'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:26.246399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-85c1c60c-0387-4320-8e3e-6361749c4411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab1c974a5ec645a7a6effa2a371a3b74'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:26.254033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-85c1c60c-0387-4320-8e3e-6361749c4411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab1c974a5ec645a7a6effa2a371a3b74'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:26.255423 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:26.271373 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-85c1c60c-0387-4320-8e3e-6361749c4411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab1c974a5ec645a7a6effa2a371a3b74'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:26.272606 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-85c1c60c-0387-4320-8e3e-6361749c4411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab1c974a5ec645a7a6effa2a371a3b74'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:26.288919 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1315/2645] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:07:26 2022] DELETE /identity/v3/users/ab1c974a5ec645a7a6effa2a371a3b74 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:26.294879 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad7737-ec45-43c6-bcb5-e791313e9278 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:26.294879 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad7737-ec45-43c6-bcb5-e791313e9278 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:26.294879 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad7737-ec45-43c6-bcb5-e791313e9278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:26.313516 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-61ad7737-ec45-43c6-bcb5-e791313e9278 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:26.314899 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-61ad7737-ec45-43c6-bcb5-e791313e9278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:26.381108 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1331/2646] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:26.391514 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1c60c-0387-4320-8e3e-6361749c4411 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:26.392997 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-85c1c60c-0387-4320-8e3e-6361749c4411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:26.456054 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1c60c-0387-4320-8e3e-6361749c4411 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:26.459766 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cae0e-208d-487c-8b93-575ac5fa4b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:26.460416 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cae0e-208d-487c-8b93-575ac5fa4b0d service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:26.460700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cae0e-208d-487c-8b93-575ac5fa4b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:26.460979 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cae0e-208d-487c-8b93-575ac5fa4b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:26.470932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cae0e-208d-487c-8b93-575ac5fa4b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:26.472372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cae0e-208d-487c-8b93-575ac5fa4b0d service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:26.482510 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1316/2647] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:26.787935 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad7737-ec45-43c6-bcb5-e791313e9278 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:26.790117 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-61ad7737-ec45-43c6-bcb5-e791313e9278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:26.803642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad7737-ec45-43c6-bcb5-e791313e9278 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:26.815798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afded93-f328-4b99-aaf8-19357132ab53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:26.816906 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afded93-f328-4b99-aaf8-19357132ab53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:26.817368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afded93-f328-4b99-aaf8-19357132ab53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:26.817891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afded93-f328-4b99-aaf8-19357132ab53 admin admin] PATH_INFO: `/v3/projects/afdf4f1e52dd4a17b939a062ddeb9b0a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:26.820876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afded93-f328-4b99-aaf8-19357132ab53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afdf4f1e52dd4a17b939a062ddeb9b0a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:26.823201 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afded93-f328-4b99-aaf8-19357132ab53 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:26.896274 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6afded93-f328-4b99-aaf8-19357132ab53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afdf4f1e52dd4a17b939a062ddeb9b0a', 'request_id': 'req-6afded93-f328-4b99-aaf8-19357132ab53'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:26.906682 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6afded93-f328-4b99-aaf8-19357132ab53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afdf4f1e52dd4a17b939a062ddeb9b0a', 'request_id': 'req-6afded93-f328-4b99-aaf8-19357132ab53'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:26.916391 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6afded93-f328-4b99-aaf8-19357132ab53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afdf4f1e52dd4a17b939a062ddeb9b0a', 'request_id': 'req-6afded93-f328-4b99-aaf8-19357132ab53'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:26.929854 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6afded93-f328-4b99-aaf8-19357132ab53 admin admin] The token cache is being invalidate because project afdf4f1e52dd4a17b939a062ddeb9b0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:07:26.930286 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6afded93-f328-4b99-aaf8-19357132ab53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:26.931107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6afded93-f328-4b99-aaf8-19357132ab53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:26.933043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1332/2648] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:07:26 2022] DELETE /identity/v3/projects/afdf4f1e52dd4a17b939a062ddeb9b0a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:26.937754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cae0e-208d-487c-8b93-575ac5fa4b0d service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:26.951866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cae0e-208d-487c-8b93-575ac5fa4b0d service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:27.015136 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add93547-d9f5-4628-913b-3c0ae36590af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:27.016380 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add93547-d9f5-4628-913b-3c0ae36590af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:27.016764 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add93547-d9f5-4628-913b-3c0ae36590af admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:27.017195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add93547-d9f5-4628-913b-3c0ae36590af admin admin] PATH_INFO: `/v3/users/e024a74ff44941e6be067b6c1dbdadbe` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:27.018720 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add93547-d9f5-4628-913b-3c0ae36590af admin admin] RBAC: Authorizing `identity:delete_user(user_id=e024a74ff44941e6be067b6c1dbdadbe)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:27.020734 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add93547-d9f5-4628-913b-3c0ae36590af admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:27.079941 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-add93547-d9f5-4628-913b-3c0ae36590af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e024a74ff44941e6be067b6c1dbdadbe'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:27.080701 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-add93547-d9f5-4628-913b-3c0ae36590af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e024a74ff44941e6be067b6c1dbdadbe'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:27.087947 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-add93547-d9f5-4628-913b-3c0ae36590af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e024a74ff44941e6be067b6c1dbdadbe'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:27.100508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-add93547-d9f5-4628-913b-3c0ae36590af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e024a74ff44941e6be067b6c1dbdadbe'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:27.101224 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-add93547-d9f5-4628-913b-3c0ae36590af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e024a74ff44941e6be067b6c1dbdadbe'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:27.104848 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:27.116344 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1317/2649] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:07:26 2022] DELETE /identity/v3/users/e024a74ff44941e6be067b6c1dbdadbe => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:07:27.539752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afded93-f328-4b99-aaf8-19357132ab53 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:27.541179 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6afded93-f328-4b99-aaf8-19357132ab53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:27.555828 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afded93-f328-4b99-aaf8-19357132ab53 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:27.637852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc4095-8e70-444b-92ff-bd9bcc7a3d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:27.638851 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc4095-8e70-444b-92ff-bd9bcc7a3d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:27.639142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc4095-8e70-444b-92ff-bd9bcc7a3d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:27.639425 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc4095-8e70-444b-92ff-bd9bcc7a3d6a admin admin] PATH_INFO: `/v3/projects/9370a62ad2bb4e7c9346b8e0ae422e16` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:27.640722 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc4095-8e70-444b-92ff-bd9bcc7a3d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9370a62ad2bb4e7c9346b8e0ae422e16)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:27.642667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc4095-8e70-444b-92ff-bd9bcc7a3d6a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:27.645314 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:27.700412 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d0fc4095-8e70-444b-92ff-bd9bcc7a3d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9370a62ad2bb4e7c9346b8e0ae422e16', 'request_id': 'req-d0fc4095-8e70-444b-92ff-bd9bcc7a3d6a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:27.708780 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d0fc4095-8e70-444b-92ff-bd9bcc7a3d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9370a62ad2bb4e7c9346b8e0ae422e16', 'request_id': 'req-d0fc4095-8e70-444b-92ff-bd9bcc7a3d6a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:27.716394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d0fc4095-8e70-444b-92ff-bd9bcc7a3d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9370a62ad2bb4e7c9346b8e0ae422e16', 'request_id': 'req-d0fc4095-8e70-444b-92ff-bd9bcc7a3d6a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:27.728364 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d0fc4095-8e70-444b-92ff-bd9bcc7a3d6a admin admin] The token cache is being invalidate because project 9370a62ad2bb4e7c9346b8e0ae422e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:07:27.728752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d0fc4095-8e70-444b-92ff-bd9bcc7a3d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:27.729324 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d0fc4095-8e70-444b-92ff-bd9bcc7a3d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:27.731664 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1333/2650] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:07:27 2022] DELETE /identity/v3/projects/9370a62ad2bb4e7c9346b8e0ae422e16 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:27.736817 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb1848-cf46-4779-b17c-3ee05b97910f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:27.737141 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb1848-cf46-4779-b17c-3ee05b97910f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:27.737442 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb1848-cf46-4779-b17c-3ee05b97910f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:27.755505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-67bb1848-cf46-4779-b17c-3ee05b97910f None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:27.756634 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-67bb1848-cf46-4779-b17c-3ee05b97910f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:27.819313 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1318/2651] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:27.828738 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed7ae5-ed9d-4ff4-a942-f027e8f82e43 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:27.829017 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed7ae5-ed9d-4ff4-a942-f027e8f82e43 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:27.829342 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed7ae5-ed9d-4ff4-a942-f027e8f82e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:27.849242 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d7ed7ae5-ed9d-4ff4-a942-f027e8f82e43 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:27.850610 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d7ed7ae5-ed9d-4ff4-a942-f027e8f82e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:27.859430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1334/2652] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:07:27.863898 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb1848-cf46-4779-b17c-3ee05b97910f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:27.874454 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb1848-cf46-4779-b17c-3ee05b97910f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:27.879285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6384abf-1944-40b3-b717-9b43fe7ee376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:27.879935 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6384abf-1944-40b3-b717-9b43fe7ee376 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:27.880255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6384abf-1944-40b3-b717-9b43fe7ee376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:27.880575 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6384abf-1944-40b3-b717-9b43fe7ee376 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:27.881356 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6384abf-1944-40b3-b717-9b43fe7ee376 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:07:27.881701 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6384abf-1944-40b3-b717-9b43fe7ee376 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:27.884077 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6384abf-1944-40b3-b717-9b43fe7ee376 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:27.897056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1319/2653] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:07:27.901613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed7ae5-ed9d-4ff4-a942-f027e8f82e43 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:27.912672 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed7ae5-ed9d-4ff4-a942-f027e8f82e43 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:27.917732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70e637-670d-4e03-baa0-d6bf5fffeb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:27.918562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70e637-670d-4e03-baa0-d6bf5fffeb45 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:27.919017 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70e637-670d-4e03-baa0-d6bf5fffeb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:27.919308 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70e637-670d-4e03-baa0-d6bf5fffeb45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:27.919998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb70e637-670d-4e03-baa0-d6bf5fffeb45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:27.922062 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb70e637-670d-4e03-baa0-d6bf5fffeb45 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:27.951537 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1335/2654] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:07:27 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:07:27.955045 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6384abf-1944-40b3-b717-9b43fe7ee376 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:27.965134 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6384abf-1944-40b3-b717-9b43fe7ee376 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:28.022874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e87da-32e3-4df9-9e89-f4ee6bd75fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:28.023457 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e87da-32e3-4df9-9e89-f4ee6bd75fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:28.024045 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e87da-32e3-4df9-9e89-f4ee6bd75fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:28.024330 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e87da-32e3-4df9-9e89-f4ee6bd75fe7 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:28.025063 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e87da-32e3-4df9-9e89-f4ee6bd75fe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:28.027015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e87da-32e3-4df9-9e89-f4ee6bd75fe7 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:28.047746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1320/2655] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:07:27 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:07:28.051817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70e637-670d-4e03-baa0-d6bf5fffeb45 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:28.062404 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70e637-670d-4e03-baa0-d6bf5fffeb45 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:28.067057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46215a60-c065-4f1b-b274-d85916a46aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:28.067739 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46215a60-c065-4f1b-b274-d85916a46aee admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:28.068019 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46215a60-c065-4f1b-b274-d85916a46aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:28.068292 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46215a60-c065-4f1b-b274-d85916a46aee admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:28.069011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46215a60-c065-4f1b-b274-d85916a46aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:28.070614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46215a60-c065-4f1b-b274-d85916a46aee admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:28.081749 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1336/2656] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:28.085687 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e87da-32e3-4df9-9e89-f4ee6bd75fe7 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:28.097211 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e87da-32e3-4df9-9e89-f4ee6bd75fe7 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:28.101892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc52a1-540d-45eb-b329-e2a448cb6026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:28.102563 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc52a1-540d-45eb-b329-e2a448cb6026 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:28.102873 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc52a1-540d-45eb-b329-e2a448cb6026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:28.103151 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc52a1-540d-45eb-b329-e2a448cb6026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:28.103866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc52a1-540d-45eb-b329-e2a448cb6026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:28.105447 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc52a1-540d-45eb-b329-e2a448cb6026 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:28.119969 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1321/2657] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:28.121152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46215a60-c065-4f1b-b274-d85916a46aee admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:28.134143 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46215a60-c065-4f1b-b274-d85916a46aee admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:28.139443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066512e7-8ce0-4bdf-9c2a-2d1375b27ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:28.140327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066512e7-8ce0-4bdf-9c2a-2d1375b27ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:28.140617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066512e7-8ce0-4bdf-9c2a-2d1375b27ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:28.140891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066512e7-8ce0-4bdf-9c2a-2d1375b27ef0 admin admin] PATH_INFO: `/v3/projects/5307924397b84960bdd7527ef80c4ea5/users/161284dbb54b477b9c6f42f1d1a8fd92/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:28.156337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066512e7-8ce0-4bdf-9c2a-2d1375b27ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5307924397b84960bdd7527ef80c4ea5, user_id=161284dbb54b477b9c6f42f1d1a8fd92, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:28.159034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066512e7-8ce0-4bdf-9c2a-2d1375b27ef0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:28.172888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1337/2658] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:07:28 2022] PUT /identity/v3/projects/5307924397b84960bdd7527ef80c4ea5/users/161284dbb54b477b9c6f42f1d1a8fd92/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:28.183156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3aec0-e8bc-4aa2-b9c0-8f274f06ffa8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:28.183461 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3aec0-e8bc-4aa2-b9c0-8f274f06ffa8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:28.183747 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3aec0-e8bc-4aa2-b9c0-8f274f06ffa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:28.211754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-3ba3aec0-e8bc-4aa2-b9c0-8f274f06ffa8 None None] MFA Rules not processed for user `161284dbb54b477b9c6f42f1d1a8fd92`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:28.212982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3ba3aec0-e8bc-4aa2-b9c0-8f274f06ffa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:28.261119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:28.281217 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1322/2659] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:28.285807 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066512e7-8ce0-4bdf-9c2a-2d1375b27ef0 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:28.350926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066512e7-8ce0-4bdf-9c2a-2d1375b27ef0 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:28.355518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a042d0-d80c-451f-bbd1-ec51faaf9e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:28.356888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a042d0-d80c-451f-bbd1-ec51faaf9e16 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:28.357344 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a042d0-d80c-451f-bbd1-ec51faaf9e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:28.357681 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a042d0-d80c-451f-bbd1-ec51faaf9e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:28.358391 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a042d0-d80c-451f-bbd1-ec51faaf9e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:28.360436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a042d0-d80c-451f-bbd1-ec51faaf9e16 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:28.385521 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1338/2660] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:07:28 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:07:28.389582 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3aec0-e8bc-4aa2-b9c0-8f274f06ffa8 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:28.400673 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3aec0-e8bc-4aa2-b9c0-8f274f06ffa8 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:28.457125 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c073ca-473d-42b1-b0bd-9e79e81e0ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:28.457796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c073ca-473d-42b1-b0bd-9e79e81e0ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:28.458083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c073ca-473d-42b1-b0bd-9e79e81e0ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:28.458363 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c073ca-473d-42b1-b0bd-9e79e81e0ff8 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:28.459108 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c073ca-473d-42b1-b0bd-9e79e81e0ff8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:28.461059 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c073ca-473d-42b1-b0bd-9e79e81e0ff8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:28.483129 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1323/2661] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:07:28 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:07:28.487141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a042d0-d80c-451f-bbd1-ec51faaf9e16 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:28.498043 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a042d0-d80c-451f-bbd1-ec51faaf9e16 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:28.502884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c3a40-6cc6-4e01-88d0-a465ec97ea6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:28.503581 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c3a40-6cc6-4e01-88d0-a465ec97ea6d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:28.503871 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c3a40-6cc6-4e01-88d0-a465ec97ea6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:28.504149 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c3a40-6cc6-4e01-88d0-a465ec97ea6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:28.504907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c3a40-6cc6-4e01-88d0-a465ec97ea6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:28.506517 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c3a40-6cc6-4e01-88d0-a465ec97ea6d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:28.517959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1339/2662] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:28.522032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c073ca-473d-42b1-b0bd-9e79e81e0ff8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:28.533867 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c073ca-473d-42b1-b0bd-9e79e81e0ff8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:28.538451 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085e2ab-60ad-45fc-ab2a-e0a22f508f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:28.539132 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085e2ab-60ad-45fc-ab2a-e0a22f508f23 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:28.539418 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085e2ab-60ad-45fc-ab2a-e0a22f508f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:28.539696 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085e2ab-60ad-45fc-ab2a-e0a22f508f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:28.540410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085e2ab-60ad-45fc-ab2a-e0a22f508f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:28.541998 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085e2ab-60ad-45fc-ab2a-e0a22f508f23 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:28.552574 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1324/2663] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:28.556812 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c3a40-6cc6-4e01-88d0-a465ec97ea6d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:28.562133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:28.568654 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c3a40-6cc6-4e01-88d0-a465ec97ea6d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:28.575317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079ab55-b69a-4a34-91f9-32a08b46ea94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:28.576063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079ab55-b69a-4a34-91f9-32a08b46ea94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:28.576344 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079ab55-b69a-4a34-91f9-32a08b46ea94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:28.576618 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079ab55-b69a-4a34-91f9-32a08b46ea94 admin admin] PATH_INFO: `/v3/projects/6d248f622e6a422699415ab5d2d1d10f/users/28ac434995794d768b3e80e9016502b2/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:28.591858 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079ab55-b69a-4a34-91f9-32a08b46ea94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d248f622e6a422699415ab5d2d1d10f, user_id=28ac434995794d768b3e80e9016502b2, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:28.594551 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079ab55-b69a-4a34-91f9-32a08b46ea94 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:28.607990 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1340/2664] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:07:28 2022] PUT /identity/v3/projects/6d248f622e6a422699415ab5d2d1d10f/users/28ac434995794d768b3e80e9016502b2/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:28.619295 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6888fbc-2e19-4b22-a992-28e06f2a1c31 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:28.619592 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6888fbc-2e19-4b22-a992-28e06f2a1c31 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:28.619878 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6888fbc-2e19-4b22-a992-28e06f2a1c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:28.648474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e6888fbc-2e19-4b22-a992-28e06f2a1c31 None None] MFA Rules not processed for user `28ac434995794d768b3e80e9016502b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:28.649947 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e6888fbc-2e19-4b22-a992-28e06f2a1c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:28.718096 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1325/2665] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:28.722722 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079ab55-b69a-4a34-91f9-32a08b46ea94 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:28.792441 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079ab55-b69a-4a34-91f9-32a08b46ea94 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:28.797677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019e9d4-30e9-47cf-b255-c4d450512e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:28.798354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019e9d4-30e9-47cf-b255-c4d450512e69 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:28.798720 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019e9d4-30e9-47cf-b255-c4d450512e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:28.798964 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019e9d4-30e9-47cf-b255-c4d450512e69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:28.799700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019e9d4-30e9-47cf-b255-c4d450512e69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:28.801647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019e9d4-30e9-47cf-b255-c4d450512e69 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:28.831427 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1341/2666] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:07:28 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:07:28.835458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6888fbc-2e19-4b22-a992-28e06f2a1c31 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:28.847119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6888fbc-2e19-4b22-a992-28e06f2a1c31 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:28.917653 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d730a-a1f6-4265-a952-10570f618457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:28.918215 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d730a-a1f6-4265-a952-10570f618457 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:28.918513 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d730a-a1f6-4265-a952-10570f618457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:28.918828 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d730a-a1f6-4265-a952-10570f618457 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:28.919527 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d730a-a1f6-4265-a952-10570f618457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:28.921462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d730a-a1f6-4265-a952-10570f618457 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:28.928639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:28.946629 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1326/2667] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:07:28 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:07:28.951917 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019e9d4-30e9-47cf-b255-c4d450512e69 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:28.964334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019e9d4-30e9-47cf-b255-c4d450512e69 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:28.971508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6605fc4-448c-4709-af33-f3204aff0a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:28.972369 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6605fc4-448c-4709-af33-f3204aff0a9e admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:28.972653 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6605fc4-448c-4709-af33-f3204aff0a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:28.972921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6605fc4-448c-4709-af33-f3204aff0a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:28.973667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6605fc4-448c-4709-af33-f3204aff0a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:28.975362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6605fc4-448c-4709-af33-f3204aff0a9e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:28.988326 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1342/2668] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:28.993462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d730a-a1f6-4265-a952-10570f618457 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:29.004091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d730a-a1f6-4265-a952-10570f618457 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:29.008805 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c2af6-0147-468a-856c-3fb242aee32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:29.009682 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c2af6-0147-468a-856c-3fb242aee32a admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:29.009956 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c2af6-0147-468a-856c-3fb242aee32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:29.010234 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c2af6-0147-468a-856c-3fb242aee32a admin admin] PATH_INFO: `/v3/projects/0d787132922b47fe9fc4c4b558648507/users/92c805adee6549169f89f6f9d64c2b48/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:29.025905 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c2af6-0147-468a-856c-3fb242aee32a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d787132922b47fe9fc4c4b558648507, user_id=92c805adee6549169f89f6f9d64c2b48, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:29.029301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c2af6-0147-468a-856c-3fb242aee32a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:29.042492 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1327/2669] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:07:28 2022] PUT /identity/v3/projects/0d787132922b47fe9fc4c4b558648507/users/92c805adee6549169f89f6f9d64c2b48/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:29.047535 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6605fc4-448c-4709-af33-f3204aff0a9e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:29.057502 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6605fc4-448c-4709-af33-f3204aff0a9e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:29.115816 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc98b8d-4d0a-4c48-9c9f-01e1b952408f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:29.116479 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc98b8d-4d0a-4c48-9c9f-01e1b952408f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:29.116762 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc98b8d-4d0a-4c48-9c9f-01e1b952408f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:29.117034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc98b8d-4d0a-4c48-9c9f-01e1b952408f admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:29.117747 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc98b8d-4d0a-4c48-9c9f-01e1b952408f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:29.119359 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc98b8d-4d0a-4c48-9c9f-01e1b952408f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:29.130703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1343/2670] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:29.134929 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c2af6-0147-468a-856c-3fb242aee32a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:29.145341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c2af6-0147-468a-856c-3fb242aee32a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:29.150222 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe32c59-8197-47d3-9b6f-052979f50147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:29.151767 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe32c59-8197-47d3-9b6f-052979f50147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:29.152050 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe32c59-8197-47d3-9b6f-052979f50147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:29.152327 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe32c59-8197-47d3-9b6f-052979f50147 admin admin] PATH_INFO: `/v3/projects/0d787132922b47fe9fc4c4b558648507/users/92c805adee6549169f89f6f9d64c2b48/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:29.155143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe32c59-8197-47d3-9b6f-052979f50147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d787132922b47fe9fc4c4b558648507, user_id=92c805adee6549169f89f6f9d64c2b48, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:29.158175 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe32c59-8197-47d3-9b6f-052979f50147 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:29.170183 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1328/2671] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:07:29 2022] PUT /identity/v3/projects/0d787132922b47fe9fc4c4b558648507/users/92c805adee6549169f89f6f9d64c2b48/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:07:29.180358 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27565bc6-69f0-43af-8b57-97b18f9e0a47 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:29.180676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27565bc6-69f0-43af-8b57-97b18f9e0a47 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:29.180950 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27565bc6-69f0-43af-8b57-97b18f9e0a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:29.207909 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-27565bc6-69f0-43af-8b57-97b18f9e0a47 None None] MFA Rules not processed for user `92c805adee6549169f89f6f9d64c2b48`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:29.209094 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-27565bc6-69f0-43af-8b57-97b18f9e0a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:29.249279 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:29.284670 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1344/2672] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:29.295962 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe32c59-8197-47d3-9b6f-052979f50147 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:29.297576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9fe32c59-8197-47d3-9b6f-052979f50147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:29.362649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe32c59-8197-47d3-9b6f-052979f50147 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:29.366360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6c3fe-b19c-4b47-916e-5b4034370f71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:29.367036 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6c3fe-b19c-4b47-916e-5b4034370f71 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:29.367323 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6c3fe-b19c-4b47-916e-5b4034370f71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:29.367601 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6c3fe-b19c-4b47-916e-5b4034370f71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:29.376939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6c3fe-b19c-4b47-916e-5b4034370f71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:29.378371 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6c3fe-b19c-4b47-916e-5b4034370f71 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:29.388567 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1329/2673] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:29.882194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27565bc6-69f0-43af-8b57-97b18f9e0a47 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:29.883730 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-27565bc6-69f0-43af-8b57-97b18f9e0a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:29.896399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27565bc6-69f0-43af-8b57-97b18f9e0a47 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:29.900436 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2aa6b-c118-4ecd-8050-f4689b8c99cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:29.901259 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2aa6b-c118-4ecd-8050-f4689b8c99cb service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:29.901700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2aa6b-c118-4ecd-8050-f4689b8c99cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:29.902153 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2aa6b-c118-4ecd-8050-f4689b8c99cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:29.968538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2aa6b-c118-4ecd-8050-f4689b8c99cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:29.969980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2aa6b-c118-4ecd-8050-f4689b8c99cb service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:29.978953 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1345/2674] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:07:30.095025 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6c3fe-b19c-4b47-916e-5b4034370f71 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:30.096425 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-13a6c3fe-b19c-4b47-916e-5b4034370f71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:30.157448 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6c3fe-b19c-4b47-916e-5b4034370f71 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:30.161099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fb410-a78e-4588-9370-2b572c5de7c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:30.161739 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fb410-a78e-4588-9370-2b572c5de7c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:30.162019 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fb410-a78e-4588-9370-2b572c5de7c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:30.162302 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fb410-a78e-4588-9370-2b572c5de7c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:30.164354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-643fb410-a78e-4588-9370-2b572c5de7c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:30.236863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fb410-a78e-4588-9370-2b572c5de7c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:30.238394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fb410-a78e-4588-9370-2b572c5de7c6 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:30.250868 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1330/2675] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:33.538217 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2aa6b-c118-4ecd-8050-f4689b8c99cb service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:33.543489 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-16f2aa6b-c118-4ecd-8050-f4689b8c99cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:33.560652 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2aa6b-c118-4ecd-8050-f4689b8c99cb service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:33.565304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd14bc2-c422-4cf5-a2d5-7b59f2c3bf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:33.566194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd14bc2-c422-4cf5-a2d5-7b59f2c3bf9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:33.566480 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd14bc2-c422-4cf5-a2d5-7b59f2c3bf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:33.567040 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd14bc2-c422-4cf5-a2d5-7b59f2c3bf9f admin admin] PATH_INFO: `/v3/users/04327f3d7b844ddb88cf4a81145634c4` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:33.568257 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd14bc2-c422-4cf5-a2d5-7b59f2c3bf9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=04327f3d7b844ddb88cf4a81145634c4)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:33.570185 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd14bc2-c422-4cf5-a2d5-7b59f2c3bf9f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:33.624657 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4cd14bc2-c422-4cf5-a2d5-7b59f2c3bf9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04327f3d7b844ddb88cf4a81145634c4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:33.625376 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4cd14bc2-c422-4cf5-a2d5-7b59f2c3bf9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04327f3d7b844ddb88cf4a81145634c4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:33.625980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4cd14bc2-c422-4cf5-a2d5-7b59f2c3bf9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04327f3d7b844ddb88cf4a81145634c4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:33.633133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4cd14bc2-c422-4cf5-a2d5-7b59f2c3bf9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04327f3d7b844ddb88cf4a81145634c4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:33.640790 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4cd14bc2-c422-4cf5-a2d5-7b59f2c3bf9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04327f3d7b844ddb88cf4a81145634c4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:33.658733 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1346/2676] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:07:33 2022] DELETE /identity/v3/users/04327f3d7b844ddb88cf4a81145634c4 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:33.836187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fb410-a78e-4588-9370-2b572c5de7c6 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:33.849040 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fb410-a78e-4588-9370-2b572c5de7c6 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:33.853979 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:33.935523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b7e90-1b20-43a9-a63f-abeab7c6aae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:33.936419 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b7e90-1b20-43a9-a63f-abeab7c6aae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:33.936700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b7e90-1b20-43a9-a63f-abeab7c6aae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:33.937039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b7e90-1b20-43a9-a63f-abeab7c6aae3 admin admin] PATH_INFO: `/v3/projects/057aef18c52b49a382450331694c04d6` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:33.938185 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b7e90-1b20-43a9-a63f-abeab7c6aae3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=057aef18c52b49a382450331694c04d6)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:33.940125 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b7e90-1b20-43a9-a63f-abeab7c6aae3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:33.998471 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-780b7e90-1b20-43a9-a63f-abeab7c6aae3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '057aef18c52b49a382450331694c04d6', 'request_id': 'req-780b7e90-1b20-43a9-a63f-abeab7c6aae3'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:34.000539 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-780b7e90-1b20-43a9-a63f-abeab7c6aae3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '057aef18c52b49a382450331694c04d6', 'request_id': 'req-780b7e90-1b20-43a9-a63f-abeab7c6aae3'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:34.007367 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-780b7e90-1b20-43a9-a63f-abeab7c6aae3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '057aef18c52b49a382450331694c04d6', 'request_id': 'req-780b7e90-1b20-43a9-a63f-abeab7c6aae3'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:34.019649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-780b7e90-1b20-43a9-a63f-abeab7c6aae3 admin admin] The token cache is being invalidate because project 057aef18c52b49a382450331694c04d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:07:34.020072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-780b7e90-1b20-43a9-a63f-abeab7c6aae3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:34.020832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-780b7e90-1b20-43a9-a63f-abeab7c6aae3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:34.022704 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1331/2677] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:07:33 2022] DELETE /identity/v3/projects/057aef18c52b49a382450331694c04d6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:34.028197 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4220eca-c564-47c6-a228-b159ab8bb6da None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:34.028484 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4220eca-c564-47c6-a228-b159ab8bb6da None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:34.028754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4220eca-c564-47c6-a228-b159ab8bb6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:34.051791 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b4220eca-c564-47c6-a228-b159ab8bb6da None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:34.053072 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b4220eca-c564-47c6-a228-b159ab8bb6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:34.054867 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b7e90-1b20-43a9-a63f-abeab7c6aae3 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:34.056307 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-780b7e90-1b20-43a9-a63f-abeab7c6aae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:34.096951 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:34.119737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1347/2678] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:34.124407 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b7e90-1b20-43a9-a63f-abeab7c6aae3 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:34.128371 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05419088-5a74-4aaa-9df6-c366177bb0cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:34.128529 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a3793-16a3-4e0e-9c58-6870a78e809d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:34.128819 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a3793-16a3-4e0e-9c58-6870a78e809d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:34.129107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a3793-16a3-4e0e-9c58-6870a78e809d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:34.129228 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05419088-5a74-4aaa-9df6-c366177bb0cf service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:34.129441 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05419088-5a74-4aaa-9df6-c366177bb0cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:34.129723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05419088-5a74-4aaa-9df6-c366177bb0cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:34.131564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-05419088-5a74-4aaa-9df6-c366177bb0cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:34.147336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-985a3793-16a3-4e0e-9c58-6870a78e809d None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:34.149034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-985a3793-16a3-4e0e-9c58-6870a78e809d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:34.158546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1348/2679] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:07:34.162922 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a3793-16a3-4e0e-9c58-6870a78e809d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:34.172575 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a3793-16a3-4e0e-9c58-6870a78e809d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:34.177204 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b0278-e432-4ec2-a91d-f4178a3cdcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:34.177856 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b0278-e432-4ec2-a91d-f4178a3cdcc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:34.178127 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b0278-e432-4ec2-a91d-f4178a3cdcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:34.178394 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b0278-e432-4ec2-a91d-f4178a3cdcc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:34.179150 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b0278-e432-4ec2-a91d-f4178a3cdcc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:07:34.179454 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b0278-e432-4ec2-a91d-f4178a3cdcc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:34.181266 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b0278-e432-4ec2-a91d-f4178a3cdcc2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:34.183241 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:34.195412 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1349/2680] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:07:34.200198 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b0278-e432-4ec2-a91d-f4178a3cdcc2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:34.204541 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05419088-5a74-4aaa-9df6-c366177bb0cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:34.206932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05419088-5a74-4aaa-9df6-c366177bb0cf service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:34.211201 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b0278-e432-4ec2-a91d-f4178a3cdcc2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:34.217494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1332/2681] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:07:34.226513 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a7dee-a3c1-40ff-88a9-b17b750f690e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:34.227381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a7dee-a3c1-40ff-88a9-b17b750f690e admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:34.227878 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a7dee-a3c1-40ff-88a9-b17b750f690e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:34.228386 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a7dee-a3c1-40ff-88a9-b17b750f690e admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:34.229604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a7dee-a3c1-40ff-88a9-b17b750f690e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:34.233798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a7dee-a3c1-40ff-88a9-b17b750f690e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:34.274299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1350/2682] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:07:34 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:07:34.278817 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05419088-5a74-4aaa-9df6-c366177bb0cf service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:34.290238 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05419088-5a74-4aaa-9df6-c366177bb0cf service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:34.359222 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265685a-79c3-483a-b968-8074443f050a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:34.360194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265685a-79c3-483a-b968-8074443f050a admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:34.360548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265685a-79c3-483a-b968-8074443f050a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:34.360915 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265685a-79c3-483a-b968-8074443f050a admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:34.362450 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265685a-79c3-483a-b968-8074443f050a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:34.364613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265685a-79c3-483a-b968-8074443f050a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:34.389006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1333/2683] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:07:34 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:07:34.390513 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a7dee-a3c1-40ff-88a9-b17b750f690e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:34.401476 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a7dee-a3c1-40ff-88a9-b17b750f690e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:34.407301 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548df0e0-4b41-4616-92c7-a679e057a6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:34.407971 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548df0e0-4b41-4616-92c7-a679e057a6eb admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:34.408259 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548df0e0-4b41-4616-92c7-a679e057a6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:34.408537 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548df0e0-4b41-4616-92c7-a679e057a6eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:34.409260 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548df0e0-4b41-4616-92c7-a679e057a6eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:34.410897 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548df0e0-4b41-4616-92c7-a679e057a6eb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:34.421766 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1351/2684] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:34.425820 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265685a-79c3-483a-b968-8074443f050a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:34.436579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265685a-79c3-483a-b968-8074443f050a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:34.441402 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7331f2a-5e9e-4749-bf3f-5e0c4346a246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:34.442087 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7331f2a-5e9e-4749-bf3f-5e0c4346a246 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:34.442389 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7331f2a-5e9e-4749-bf3f-5e0c4346a246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:34.442713 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7331f2a-5e9e-4749-bf3f-5e0c4346a246 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:34.443577 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7331f2a-5e9e-4749-bf3f-5e0c4346a246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:34.445259 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7331f2a-5e9e-4749-bf3f-5e0c4346a246 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:34.456047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1334/2685] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:34.460311 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548df0e0-4b41-4616-92c7-a679e057a6eb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:34.471620 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548df0e0-4b41-4616-92c7-a679e057a6eb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:34.476344 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffbd51-7e74-4622-8199-5bd033a1909d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:34.477324 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffbd51-7e74-4622-8199-5bd033a1909d admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:34.477603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffbd51-7e74-4622-8199-5bd033a1909d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:34.477885 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffbd51-7e74-4622-8199-5bd033a1909d admin admin] PATH_INFO: `/v3/projects/bf39ecb69ea7474d940ad17f8bab4874/users/55f5aba8355c436b94508156bca24ab3/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:34.496330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffbd51-7e74-4622-8199-5bd033a1909d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf39ecb69ea7474d940ad17f8bab4874, user_id=55f5aba8355c436b94508156bca24ab3, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:34.499886 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffbd51-7e74-4622-8199-5bd033a1909d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:34.515085 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1352/2686] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:07:34 2022] PUT /identity/v3/projects/bf39ecb69ea7474d940ad17f8bab4874/users/55f5aba8355c436b94508156bca24ab3/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:34.520288 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d7ac0-2876-4398-b096-af0420b4387f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:34.520577 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d7ac0-2876-4398-b096-af0420b4387f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:34.520855 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d7ac0-2876-4398-b096-af0420b4387f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:34.541104 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-880d7ac0-2876-4398-b096-af0420b4387f None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:34.549632 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-880d7ac0-2876-4398-b096-af0420b4387f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:34.618617 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1335/2687] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:34.627888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffbd51-7e74-4622-8199-5bd033a1909d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:34.630773 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-12ffbd51-7e74-4622-8199-5bd033a1909d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:34.696963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffbd51-7e74-4622-8199-5bd033a1909d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:34.701638 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea3828-5d38-428f-abeb-621c3ab03c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:34.702292 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea3828-5d38-428f-abeb-621c3ab03c73 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:34.702597 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea3828-5d38-428f-abeb-621c3ab03c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:34.702894 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea3828-5d38-428f-abeb-621c3ab03c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:34.712510 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea3828-5d38-428f-abeb-621c3ab03c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:34.713943 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea3828-5d38-428f-abeb-621c3ab03c73 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:34.724787 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1353/2688] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:36.455616 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d7ac0-2876-4398-b096-af0420b4387f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:36.458944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-880d7ac0-2876-4398-b096-af0420b4387f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:36.487741 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d7ac0-2876-4398-b096-af0420b4387f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:36.492635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f86139-9c32-4bf5-8c45-954ca7192b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:36.493512 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f86139-9c32-4bf5-8c45-954ca7192b4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:36.493799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f86139-9c32-4bf5-8c45-954ca7192b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:36.494078 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f86139-9c32-4bf5-8c45-954ca7192b4a admin admin] PATH_INFO: `/v3/users/161284dbb54b477b9c6f42f1d1a8fd92` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:36.495352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f86139-9c32-4bf5-8c45-954ca7192b4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=161284dbb54b477b9c6f42f1d1a8fd92)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:36.497379 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f86139-9c32-4bf5-8c45-954ca7192b4a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:36.572198 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-10f86139-9c32-4bf5-8c45-954ca7192b4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '161284dbb54b477b9c6f42f1d1a8fd92'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:36.573000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-10f86139-9c32-4bf5-8c45-954ca7192b4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '161284dbb54b477b9c6f42f1d1a8fd92'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:36.582298 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-10f86139-9c32-4bf5-8c45-954ca7192b4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '161284dbb54b477b9c6f42f1d1a8fd92'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:36.588105 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:36.598476 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-10f86139-9c32-4bf5-8c45-954ca7192b4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '161284dbb54b477b9c6f42f1d1a8fd92'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:36.599893 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-10f86139-9c32-4bf5-8c45-954ca7192b4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '161284dbb54b477b9c6f42f1d1a8fd92'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:36.614655 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1336/2689] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:07:36 2022] DELETE /identity/v3/users/161284dbb54b477b9c6f42f1d1a8fd92 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:36.620286 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee729a31-4f60-4afc-b855-9291c2a0841b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:36.620784 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee729a31-4f60-4afc-b855-9291c2a0841b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:36.620903 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee729a31-4f60-4afc-b855-9291c2a0841b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:36.639921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ee729a31-4f60-4afc-b855-9291c2a0841b None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:36.641154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ee729a31-4f60-4afc-b855-9291c2a0841b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:36.707825 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1354/2690] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:36.718328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f86139-9c32-4bf5-8c45-954ca7192b4a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:36.719876 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-10f86139-9c32-4bf5-8c45-954ca7192b4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:36.783680 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f86139-9c32-4bf5-8c45-954ca7192b4a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:36.787369 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8408c2ab-a88c-48f7-a86e-1005a7d49554 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:36.788028 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408c2ab-a88c-48f7-a86e-1005a7d49554 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:36.788326 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408c2ab-a88c-48f7-a86e-1005a7d49554 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:36.788605 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408c2ab-a88c-48f7-a86e-1005a7d49554 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:36.800821 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8408c2ab-a88c-48f7-a86e-1005a7d49554 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:36.802397 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8408c2ab-a88c-48f7-a86e-1005a7d49554 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:36.815844 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1337/2691] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:37.125858 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee729a31-4f60-4afc-b855-9291c2a0841b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:37.128308 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ee729a31-4f60-4afc-b855-9291c2a0841b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:37.143748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee729a31-4f60-4afc-b855-9291c2a0841b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:37.151850 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19973d6-70e4-4cfd-914d-86e5afedce70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:37.153562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19973d6-70e4-4cfd-914d-86e5afedce70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:37.153872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19973d6-70e4-4cfd-914d-86e5afedce70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:37.154105 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19973d6-70e4-4cfd-914d-86e5afedce70 admin admin] PATH_INFO: `/v3/projects/5307924397b84960bdd7527ef80c4ea5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:37.155389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19973d6-70e4-4cfd-914d-86e5afedce70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5307924397b84960bdd7527ef80c4ea5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:37.157308 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19973d6-70e4-4cfd-914d-86e5afedce70 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:37.165200 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:37.243781 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f19973d6-70e4-4cfd-914d-86e5afedce70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5307924397b84960bdd7527ef80c4ea5', 'request_id': 'req-f19973d6-70e4-4cfd-914d-86e5afedce70'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:37.254893 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f19973d6-70e4-4cfd-914d-86e5afedce70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5307924397b84960bdd7527ef80c4ea5', 'request_id': 'req-f19973d6-70e4-4cfd-914d-86e5afedce70'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:37.265887 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f19973d6-70e4-4cfd-914d-86e5afedce70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5307924397b84960bdd7527ef80c4ea5', 'request_id': 'req-f19973d6-70e4-4cfd-914d-86e5afedce70'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:37.278450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f19973d6-70e4-4cfd-914d-86e5afedce70 admin admin] The token cache is being invalidate because project 5307924397b84960bdd7527ef80c4ea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:07:37.278894 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f19973d6-70e4-4cfd-914d-86e5afedce70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:37.279683 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f19973d6-70e4-4cfd-914d-86e5afedce70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:37.281659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1355/2692] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:07:37 2022] DELETE /identity/v3/projects/5307924397b84960bdd7527ef80c4ea5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:37.285443 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8408c2ab-a88c-48f7-a86e-1005a7d49554 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:37.304150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8408c2ab-a88c-48f7-a86e-1005a7d49554 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:37.428162 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90e5ca-291e-42d3-af61-cdfd97a0da5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:37.429065 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90e5ca-291e-42d3-af61-cdfd97a0da5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:37.429349 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90e5ca-291e-42d3-af61-cdfd97a0da5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:37.429627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90e5ca-291e-42d3-af61-cdfd97a0da5f admin admin] PATH_INFO: `/v3/users/28ac434995794d768b3e80e9016502b2` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:37.431359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90e5ca-291e-42d3-af61-cdfd97a0da5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=28ac434995794d768b3e80e9016502b2)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:37.433387 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90e5ca-291e-42d3-af61-cdfd97a0da5f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:37.499700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6c90e5ca-291e-42d3-af61-cdfd97a0da5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28ac434995794d768b3e80e9016502b2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:37.501136 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6c90e5ca-291e-42d3-af61-cdfd97a0da5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28ac434995794d768b3e80e9016502b2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:37.509438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6c90e5ca-291e-42d3-af61-cdfd97a0da5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28ac434995794d768b3e80e9016502b2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:37.525508 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6c90e5ca-291e-42d3-af61-cdfd97a0da5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28ac434995794d768b3e80e9016502b2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:37.526665 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6c90e5ca-291e-42d3-af61-cdfd97a0da5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28ac434995794d768b3e80e9016502b2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:37.548150 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1338/2693] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:07:37 2022] DELETE /identity/v3/users/28ac434995794d768b3e80e9016502b2 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:37.869079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19973d6-70e4-4cfd-914d-86e5afedce70 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:37.870558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f19973d6-70e4-4cfd-914d-86e5afedce70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:37.896373 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19973d6-70e4-4cfd-914d-86e5afedce70 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:38.012839 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:38.013193 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:38.013589 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:38.013940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491 admin admin] PATH_INFO: `/v3/projects/6d248f622e6a422699415ab5d2d1d10f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:38.015601 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d248f622e6a422699415ab5d2d1d10f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:38.017640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:38.079570 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d248f622e6a422699415ab5d2d1d10f', 'request_id': 'req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:38.092609 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d248f622e6a422699415ab5d2d1d10f', 'request_id': 'req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:38.102112 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d248f622e6a422699415ab5d2d1d10f', 'request_id': 'req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:38.116686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491 admin admin] The token cache is being invalidate because project 6d248f622e6a422699415ab5d2d1d10f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:07:38.117128 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:38.118687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:38.120768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1356/2694] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:07:37 2022] DELETE /identity/v3/projects/6d248f622e6a422699415ab5d2d1d10f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:38.131716 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90e5ca-291e-42d3-af61-cdfd97a0da5f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:38.134373 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6c90e5ca-291e-42d3-af61-cdfd97a0da5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:38.150618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:38.164211 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90e5ca-291e-42d3-af61-cdfd97a0da5f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:38.237840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c1743-423f-479e-b9dd-c72a4a7e8ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:38.239093 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c1743-423f-479e-b9dd-c72a4a7e8ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:38.239478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c1743-423f-479e-b9dd-c72a4a7e8ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:38.239853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c1743-423f-479e-b9dd-c72a4a7e8ab9 admin admin] PATH_INFO: `/v3/users/92c805adee6549169f89f6f9d64c2b48` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:38.242003 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c1743-423f-479e-b9dd-c72a4a7e8ab9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92c805adee6549169f89f6f9d64c2b48)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:38.244853 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c1743-423f-479e-b9dd-c72a4a7e8ab9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:38.319805 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-222c1743-423f-479e-b9dd-c72a4a7e8ab9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92c805adee6549169f89f6f9d64c2b48'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:38.320738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-222c1743-423f-479e-b9dd-c72a4a7e8ab9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92c805adee6549169f89f6f9d64c2b48'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:38.332667 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-222c1743-423f-479e-b9dd-c72a4a7e8ab9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92c805adee6549169f89f6f9d64c2b48'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:38.350783 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-222c1743-423f-479e-b9dd-c72a4a7e8ab9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92c805adee6549169f89f6f9d64c2b48'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:38.351852 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-222c1743-423f-479e-b9dd-c72a4a7e8ab9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92c805adee6549169f89f6f9d64c2b48'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:38.376208 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1339/2695] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:07:38 2022] DELETE /identity/v3/users/92c805adee6549169f89f6f9d64c2b48 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:38.747948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:38.749486 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:38.770939 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed97b4-d3aa-479e-8de5-bd4d4c1bd491 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:38.798207 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:38.849635 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfc838-e14c-409b-a81f-c6d173671fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:38.850814 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfc838-e14c-409b-a81f-c6d173671fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:38.850957 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfc838-e14c-409b-a81f-c6d173671fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:38.851305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfc838-e14c-409b-a81f-c6d173671fe3 admin admin] PATH_INFO: `/v3/projects/0d787132922b47fe9fc4c4b558648507` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:38.852684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bfc838-e14c-409b-a81f-c6d173671fe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d787132922b47fe9fc4c4b558648507)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:38.854617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bfc838-e14c-409b-a81f-c6d173671fe3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:38.926437 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-63bfc838-e14c-409b-a81f-c6d173671fe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d787132922b47fe9fc4c4b558648507', 'request_id': 'req-63bfc838-e14c-409b-a81f-c6d173671fe3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:38.936183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-63bfc838-e14c-409b-a81f-c6d173671fe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d787132922b47fe9fc4c4b558648507', 'request_id': 'req-63bfc838-e14c-409b-a81f-c6d173671fe3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:38.950292 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-63bfc838-e14c-409b-a81f-c6d173671fe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d787132922b47fe9fc4c4b558648507', 'request_id': 'req-63bfc838-e14c-409b-a81f-c6d173671fe3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:38.971508 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-63bfc838-e14c-409b-a81f-c6d173671fe3 admin admin] The token cache is being invalidate because project 0d787132922b47fe9fc4c4b558648507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:07:38.971803 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-63bfc838-e14c-409b-a81f-c6d173671fe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:38.972657 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-63bfc838-e14c-409b-a81f-c6d173671fe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:38.974729 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1357/2696] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:07:38 2022] DELETE /identity/v3/projects/0d787132922b47fe9fc4c4b558648507 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:38.981829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836d1df-5791-4c6c-b32d-a5312f3eb1ae None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:38.983272 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836d1df-5791-4c6c-b32d-a5312f3eb1ae None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:38.983736 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836d1df-5791-4c6c-b32d-a5312f3eb1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:39.009162 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d836d1df-5791-4c6c-b32d-a5312f3eb1ae None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:39.010857 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d836d1df-5791-4c6c-b32d-a5312f3eb1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:39.098734 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1340/2697] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:39.107926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611d604-9469-42a9-841b-cd5702608640 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:39.108210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611d604-9469-42a9-841b-cd5702608640 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:39.108716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611d604-9469-42a9-841b-cd5702608640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:39.131224 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2611d604-9469-42a9-841b-cd5702608640 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:39.133270 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2611d604-9469-42a9-841b-cd5702608640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:39.143303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1358/2698] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:39.148398 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836d1df-5791-4c6c-b32d-a5312f3eb1ae None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:39.164796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836d1df-5791-4c6c-b32d-a5312f3eb1ae None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:39.166851 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83b8a6-5ff4-43fe-9dc7-3c55e358c9dd None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:39.167047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7931f-e6df-412b-b82f-e430344ac41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:39.167325 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83b8a6-5ff4-43fe-9dc7-3c55e358c9dd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:39.167640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83b8a6-5ff4-43fe-9dc7-3c55e358c9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:39.167973 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7931f-e6df-412b-b82f-e430344ac41c admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:39.168443 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7931f-e6df-412b-b82f-e430344ac41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:39.168789 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7931f-e6df-412b-b82f-e430344ac41c admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:39.169753 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7931f-e6df-412b-b82f-e430344ac41c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:07:39.170153 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7931f-e6df-412b-b82f-e430344ac41c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:39.171814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7931f-e6df-412b-b82f-e430344ac41c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:39.187387 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1341/2699] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:07:39.192020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7931f-e6df-412b-b82f-e430344ac41c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:39.202278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ab83b8a6-5ff4-43fe-9dc7-3c55e358c9dd None None] MFA Rules not processed for user `55f5aba8355c436b94508156bca24ab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:39.203512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ab83b8a6-5ff4-43fe-9dc7-3c55e358c9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:39.203728 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7931f-e6df-412b-b82f-e430344ac41c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:39.209062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9056c4-3edf-4ff1-9f2f-44abd7a67310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:39.209905 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9056c4-3edf-4ff1-9f2f-44abd7a67310 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:39.209905 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9056c4-3edf-4ff1-9f2f-44abd7a67310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:39.210222 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9056c4-3edf-4ff1-9f2f-44abd7a67310 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:39.211335 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9056c4-3edf-4ff1-9f2f-44abd7a67310 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:39.213164 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9056c4-3edf-4ff1-9f2f-44abd7a67310 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:39.218497 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:39.248514 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1342/2700] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:07:39 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:07:39.259587 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9056c4-3edf-4ff1-9f2f-44abd7a67310 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:39.291830 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1359/2701] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:39.307303 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83b8a6-5ff4-43fe-9dc7-3c55e358c9dd None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:39.310598 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ab83b8a6-5ff4-43fe-9dc7-3c55e358c9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:39.359422 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9056c4-3edf-4ff1-9f2f-44abd7a67310 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:39.361532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:39.365099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca872ee2-7313-4507-b5ce-ed862dc3c6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:39.365783 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca872ee2-7313-4507-b5ce-ed862dc3c6c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:39.366063 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca872ee2-7313-4507-b5ce-ed862dc3c6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:39.366633 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca872ee2-7313-4507-b5ce-ed862dc3c6c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:39.367221 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca872ee2-7313-4507-b5ce-ed862dc3c6c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:39.369178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca872ee2-7313-4507-b5ce-ed862dc3c6c2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:39.392315 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1343/2702] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:07:39 2022] POST /identity/v3/users => generated 282 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:07:39.396355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca872ee2-7313-4507-b5ce-ed862dc3c6c2 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:39.407148 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca872ee2-7313-4507-b5ce-ed862dc3c6c2 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:39.412742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ffd48-86b1-4b18-9ec7-96a4187cbaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:39.413400 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ffd48-86b1-4b18-9ec7-96a4187cbaae admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:39.414258 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ffd48-86b1-4b18-9ec7-96a4187cbaae admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:39.414393 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83b8a6-5ff4-43fe-9dc7-3c55e358c9dd None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:39.414632 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ffd48-86b1-4b18-9ec7-96a4187cbaae admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:39.414738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ffd48-86b1-4b18-9ec7-96a4187cbaae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:39.416278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ffd48-86b1-4b18-9ec7-96a4187cbaae admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:39.431829 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1344/2703] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:07:39.432630 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ffd48-86b1-4b18-9ec7-96a4187cbaae admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:39.443266 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ffd48-86b1-4b18-9ec7-96a4187cbaae admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:39.448127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab056e8f-4d95-4417-9a83-5a8c0634b3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:39.448860 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab056e8f-4d95-4417-9a83-5a8c0634b3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:39.449232 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab056e8f-4d95-4417-9a83-5a8c0634b3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:39.449770 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab056e8f-4d95-4417-9a83-5a8c0634b3b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:39.450721 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab056e8f-4d95-4417-9a83-5a8c0634b3b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:39.452439 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab056e8f-4d95-4417-9a83-5a8c0634b3b8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:39.463993 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1345/2704] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:39.468708 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab056e8f-4d95-4417-9a83-5a8c0634b3b8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:39.480220 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab056e8f-4d95-4417-9a83-5a8c0634b3b8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:39.485848 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694f445-a273-44f0-84b0-b306be6a6837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:39.486875 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694f445-a273-44f0-84b0-b306be6a6837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:39.487160 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694f445-a273-44f0-84b0-b306be6a6837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:39.487642 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694f445-a273-44f0-84b0-b306be6a6837 admin admin] PATH_INFO: `/v3/projects/b8d7503451284a71a415d923342be573/users/6c4f296f943248c1b5c38d0fa72fd373/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:39.496115 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7e181-e7d5-4fec-aff7-663afc15f13b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:39.497673 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7e181-e7d5-4fec-aff7-663afc15f13b service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:39.497967 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7e181-e7d5-4fec-aff7-663afc15f13b service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:39.498246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7e181-e7d5-4fec-aff7-663afc15f13b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:39.504629 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694f445-a273-44f0-84b0-b306be6a6837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8d7503451284a71a415d923342be573, user_id=6c4f296f943248c1b5c38d0fa72fd373, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:39.507320 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694f445-a273-44f0-84b0-b306be6a6837 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:39.534675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1346/2705] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:07:39 2022] PUT /identity/v3/projects/b8d7503451284a71a415d923342be573/users/6c4f296f943248c1b5c38d0fa72fd373/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:07:39.539418 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c2691-3bfc-4ccb-9230-34fd362d176e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:39.539980 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c2691-3bfc-4ccb-9230-34fd362d176e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:39.540474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c2691-3bfc-4ccb-9230-34fd362d176e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:39.576392 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-235c2691-3bfc-4ccb-9230-34fd362d176e None None] MFA Rules not processed for user `6c4f296f943248c1b5c38d0fa72fd373`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:39.578247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-235c2691-3bfc-4ccb-9230-34fd362d176e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:39.586849 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7e181-e7d5-4fec-aff7-663afc15f13b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:39.589004 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7e181-e7d5-4fec-aff7-663afc15f13b service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:39.601139 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1360/2706] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:07:39.662262 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1347/2707] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:39 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:39.666909 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7e181-e7d5-4fec-aff7-663afc15f13b service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:39.677955 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7e181-e7d5-4fec-aff7-663afc15f13b service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:39.752697 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a8ffb-416f-4693-80b4-e2d25111e9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:39.753873 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a8ffb-416f-4693-80b4-e2d25111e9aa admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:39.754119 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a8ffb-416f-4693-80b4-e2d25111e9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:39.754401 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a8ffb-416f-4693-80b4-e2d25111e9aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:39.755150 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a8ffb-416f-4693-80b4-e2d25111e9aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:39.759507 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a8ffb-416f-4693-80b4-e2d25111e9aa admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:39.786087 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:39.798326 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1361/2708] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:07:39 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:07:39.799574 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c2691-3bfc-4ccb-9230-34fd362d176e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:39.814332 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c2691-3bfc-4ccb-9230-34fd362d176e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:39.834495 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:39.884877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084c192-1ad7-42ac-8ac6-1f15a9416fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:39.885772 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084c192-1ad7-42ac-8ac6-1f15a9416fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:39.886119 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084c192-1ad7-42ac-8ac6-1f15a9416fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:39.886460 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084c192-1ad7-42ac-8ac6-1f15a9416fe3 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:39.887447 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084c192-1ad7-42ac-8ac6-1f15a9416fe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:39.889455 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084c192-1ad7-42ac-8ac6-1f15a9416fe3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:39.911851 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1348/2709] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:07:39 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:07:39.915767 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a8ffb-416f-4693-80b4-e2d25111e9aa admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:39.927161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a8ffb-416f-4693-80b4-e2d25111e9aa admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:39.931891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed6029-2d61-47e0-87bf-abfd0ef6207c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:39.932557 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed6029-2d61-47e0-87bf-abfd0ef6207c admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:39.932835 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed6029-2d61-47e0-87bf-abfd0ef6207c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:39.933104 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed6029-2d61-47e0-87bf-abfd0ef6207c admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:39.933860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed6029-2d61-47e0-87bf-abfd0ef6207c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:39.936175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed6029-2d61-47e0-87bf-abfd0ef6207c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:39.949005 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1362/2710] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:39.952849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084c192-1ad7-42ac-8ac6-1f15a9416fe3 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:39.964246 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084c192-1ad7-42ac-8ac6-1f15a9416fe3 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:39.970005 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef90b3-bd57-43c0-8e49-3c530c3d7d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:39.970957 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef90b3-bd57-43c0-8e49-3c530c3d7d3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:39.971244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef90b3-bd57-43c0-8e49-3c530c3d7d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:39.971520 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef90b3-bd57-43c0-8e49-3c530c3d7d3a admin admin] PATH_INFO: `/v3/projects/e130d20bf7d34add86c1017682fa105d/users/cf81f2bde28e4c2388a07b3147b9f212/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:39.989607 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef90b3-bd57-43c0-8e49-3c530c3d7d3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e130d20bf7d34add86c1017682fa105d, user_id=cf81f2bde28e4c2388a07b3147b9f212, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:39.992307 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef90b3-bd57-43c0-8e49-3c530c3d7d3a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:40.007561 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1349/2711] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:07:39 2022] PUT /identity/v3/projects/e130d20bf7d34add86c1017682fa105d/users/cf81f2bde28e4c2388a07b3147b9f212/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:07:40.011412 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed6029-2d61-47e0-87bf-abfd0ef6207c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:40.022734 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed6029-2d61-47e0-87bf-abfd0ef6207c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:40.084637 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52e76b-5a6c-4e96-98ba-b76e18208878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:40.085355 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52e76b-5a6c-4e96-98ba-b76e18208878 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:40.085648 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52e76b-5a6c-4e96-98ba-b76e18208878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:40.085944 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52e76b-5a6c-4e96-98ba-b76e18208878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:40.086766 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52e76b-5a6c-4e96-98ba-b76e18208878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:40.088431 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52e76b-5a6c-4e96-98ba-b76e18208878 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:40.099829 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1363/2712] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:40.104112 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef90b3-bd57-43c0-8e49-3c530c3d7d3a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:40.116267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef90b3-bd57-43c0-8e49-3c530c3d7d3a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:40.121282 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d80c5b-3844-4e7f-b3f1-78ad8fb71485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:40.123480 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d80c5b-3844-4e7f-b3f1-78ad8fb71485 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:40.123764 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d80c5b-3844-4e7f-b3f1-78ad8fb71485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:40.124067 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d80c5b-3844-4e7f-b3f1-78ad8fb71485 admin admin] PATH_INFO: `/v3/projects/e130d20bf7d34add86c1017682fa105d/users/cf81f2bde28e4c2388a07b3147b9f212/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:40.126165 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d80c5b-3844-4e7f-b3f1-78ad8fb71485 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e130d20bf7d34add86c1017682fa105d, user_id=cf81f2bde28e4c2388a07b3147b9f212, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:40.128782 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d80c5b-3844-4e7f-b3f1-78ad8fb71485 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:40.140489 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1350/2713] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:07:40 2022] PUT /identity/v3/projects/e130d20bf7d34add86c1017682fa105d/users/cf81f2bde28e4c2388a07b3147b9f212/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:40.151377 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6d8f5-a690-4265-a16c-279ca3beae67 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:40.151670 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6d8f5-a690-4265-a16c-279ca3beae67 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:40.151947 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6d8f5-a690-4265-a16c-279ca3beae67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:40.184209 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-58e6d8f5-a690-4265-a16c-279ca3beae67 None None] MFA Rules not processed for user `cf81f2bde28e4c2388a07b3147b9f212`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:40.185881 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-58e6d8f5-a690-4265-a16c-279ca3beae67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:40.281389 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1364/2714] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:40 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:40.294372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d80c5b-3844-4e7f-b3f1-78ad8fb71485 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:40.296265 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-89d80c5b-3844-4e7f-b3f1-78ad8fb71485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:40.359103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d80c5b-3844-4e7f-b3f1-78ad8fb71485 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:40.363316 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e45a8c-23e7-4eaf-b5d8-e07b6df32c33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:40.364050 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e45a8c-23e7-4eaf-b5d8-e07b6df32c33 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:40.364342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e45a8c-23e7-4eaf-b5d8-e07b6df32c33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:40.364617 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e45a8c-23e7-4eaf-b5d8-e07b6df32c33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:40.375450 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e45a8c-23e7-4eaf-b5d8-e07b6df32c33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:40.376989 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e45a8c-23e7-4eaf-b5d8-e07b6df32c33 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:40.388385 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1351/2715] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:40 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:40.474073 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6d8f5-a690-4265-a16c-279ca3beae67 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:40.484991 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6d8f5-a690-4265-a16c-279ca3beae67 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:40.489123 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb4ba2-3d99-402d-9b6f-7589268cac25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:40.489878 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb4ba2-3d99-402d-9b6f-7589268cac25 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:40.490239 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb4ba2-3d99-402d-9b6f-7589268cac25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:40.490648 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb4ba2-3d99-402d-9b6f-7589268cac25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:40.553336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb4ba2-3d99-402d-9b6f-7589268cac25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:40.554865 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb4ba2-3d99-402d-9b6f-7589268cac25 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:40.565845 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1365/2716] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:40 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:42.007558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af116a35-9b44-40c2-9305-f484599891dc None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:42.008068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af116a35-9b44-40c2-9305-f484599891dc None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:42.008177 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af116a35-9b44-40c2-9305-f484599891dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:42.028721 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-af116a35-9b44-40c2-9305-f484599891dc None None] MFA Rules not processed for user `55f5aba8355c436b94508156bca24ab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:42.030308 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-af116a35-9b44-40c2-9305-f484599891dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:42.101274 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1352/2717] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:44.870885 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb4ba2-3d99-402d-9b6f-7589268cac25 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:44.881141 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb4ba2-3d99-402d-9b6f-7589268cac25 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:44.885084 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5741ac-ce9b-4ce4-a73b-7ad5432e314c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:44.885792 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5741ac-ce9b-4ce4-a73b-7ad5432e314c service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:44.886116 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5741ac-ce9b-4ce4-a73b-7ad5432e314c service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:44.886413 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5741ac-ce9b-4ce4-a73b-7ad5432e314c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:44.888769 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cd5741ac-ce9b-4ce4-a73b-7ad5432e314c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:44.894805 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:44.981350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5741ac-ce9b-4ce4-a73b-7ad5432e314c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:44.983272 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5741ac-ce9b-4ce4-a73b-7ad5432e314c service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:44.992959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1366/2718] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:45.596657 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af116a35-9b44-40c2-9305-f484599891dc None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:45.620244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:45.680371 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af116a35-9b44-40c2-9305-f484599891dc None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:45.685927 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74040f4-c030-4db7-b35d-3f1b894d6cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:45.687015 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74040f4-c030-4db7-b35d-3f1b894d6cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:45.687375 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74040f4-c030-4db7-b35d-3f1b894d6cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:45.687708 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74040f4-c030-4db7-b35d-3f1b894d6cf7 admin admin] PATH_INFO: `/v3/users/6c4f296f943248c1b5c38d0fa72fd373` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:45.688973 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74040f4-c030-4db7-b35d-3f1b894d6cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c4f296f943248c1b5c38d0fa72fd373)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:45.691070 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74040f4-c030-4db7-b35d-3f1b894d6cf7 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:45.744833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b74040f4-c030-4db7-b35d-3f1b894d6cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c4f296f943248c1b5c38d0fa72fd373'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:45.745504 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b74040f4-c030-4db7-b35d-3f1b894d6cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c4f296f943248c1b5c38d0fa72fd373'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:45.753113 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b74040f4-c030-4db7-b35d-3f1b894d6cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c4f296f943248c1b5c38d0fa72fd373'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:45.766979 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b74040f4-c030-4db7-b35d-3f1b894d6cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c4f296f943248c1b5c38d0fa72fd373'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:45.767665 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b74040f4-c030-4db7-b35d-3f1b894d6cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c4f296f943248c1b5c38d0fa72fd373'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:45.781463 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1353/2719] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:07:45 2022] DELETE /identity/v3/users/6c4f296f943248c1b5c38d0fa72fd373 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:45.785866 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec14578-a1f0-4dc4-9a7a-05b98182d1f8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:45.786289 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec14578-a1f0-4dc4-9a7a-05b98182d1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:45.786581 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec14578-a1f0-4dc4-9a7a-05b98182d1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:45.803773 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9ec14578-a1f0-4dc4-9a7a-05b98182d1f8 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:45.804968 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9ec14578-a1f0-4dc4-9a7a-05b98182d1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:45.865582 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1367/2720] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:45.875085 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74040f4-c030-4db7-b35d-3f1b894d6cf7 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:45.876484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b74040f4-c030-4db7-b35d-3f1b894d6cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:45.937514 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74040f4-c030-4db7-b35d-3f1b894d6cf7 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:45.941374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec732f-d016-43b3-ad81-79ab2e8ba847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:45.942070 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec732f-d016-43b3-ad81-79ab2e8ba847 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:45.942372 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec732f-d016-43b3-ad81-79ab2e8ba847 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:45.942707 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec732f-d016-43b3-ad81-79ab2e8ba847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:45.953361 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec732f-d016-43b3-ad81-79ab2e8ba847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:45.955006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec732f-d016-43b3-ad81-79ab2e8ba847 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:45.966187 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1354/2721] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:46.256296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec14578-a1f0-4dc4-9a7a-05b98182d1f8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:46.257782 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9ec14578-a1f0-4dc4-9a7a-05b98182d1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:46.271001 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec14578-a1f0-4dc4-9a7a-05b98182d1f8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:46.275905 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:46.276784 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:46.277182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:46.278091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619 admin admin] PATH_INFO: `/v3/projects/b8d7503451284a71a415d923342be573` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:46.280088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8d7503451284a71a415d923342be573)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:46.281890 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:46.332700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8d7503451284a71a415d923342be573', 'request_id': 'req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:46.342715 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8d7503451284a71a415d923342be573', 'request_id': 'req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:46.349693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8d7503451284a71a415d923342be573', 'request_id': 'req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:46.361400 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619 admin admin] The token cache is being invalidate because project b8d7503451284a71a415d923342be573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:07:46.361687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:46.362338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:46.363963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1368/2722] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:07:46 2022] DELETE /identity/v3/projects/b8d7503451284a71a415d923342be573 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:46.369637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec732f-d016-43b3-ad81-79ab2e8ba847 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:46.379523 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec732f-d016-43b3-ad81-79ab2e8ba847 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:46.423593 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:46.441571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe3207-46f6-45b9-9727-ab8abc73ad44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:46.442750 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe3207-46f6-45b9-9727-ab8abc73ad44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:46.443039 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe3207-46f6-45b9-9727-ab8abc73ad44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:46.443319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe3207-46f6-45b9-9727-ab8abc73ad44 admin admin] PATH_INFO: `/v3/users/cf81f2bde28e4c2388a07b3147b9f212` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:46.444679 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe3207-46f6-45b9-9727-ab8abc73ad44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf81f2bde28e4c2388a07b3147b9f212)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:46.446864 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe3207-46f6-45b9-9727-ab8abc73ad44 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:46.541254 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3efe3207-46f6-45b9-9727-ab8abc73ad44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf81f2bde28e4c2388a07b3147b9f212'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:46.541923 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3efe3207-46f6-45b9-9727-ab8abc73ad44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf81f2bde28e4c2388a07b3147b9f212'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:46.551048 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3efe3207-46f6-45b9-9727-ab8abc73ad44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf81f2bde28e4c2388a07b3147b9f212'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:46.563672 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3efe3207-46f6-45b9-9727-ab8abc73ad44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf81f2bde28e4c2388a07b3147b9f212'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:46.564384 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3efe3207-46f6-45b9-9727-ab8abc73ad44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf81f2bde28e4c2388a07b3147b9f212'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:46.578237 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1355/2723] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:07:46 2022] DELETE /identity/v3/users/cf81f2bde28e4c2388a07b3147b9f212 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:07:46.856470 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:46.858194 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:46.871663 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12cd07-09c3-41ad-8eb4-8c74d5c8f619 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:46.901486 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:46.935132 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4acc23-344a-4936-abd0-503a2bdcea45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:46.936001 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4acc23-344a-4936-abd0-503a2bdcea45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:46.936286 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4acc23-344a-4936-abd0-503a2bdcea45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:46.936565 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4acc23-344a-4936-abd0-503a2bdcea45 admin admin] PATH_INFO: `/v3/projects/e130d20bf7d34add86c1017682fa105d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:46.937746 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4acc23-344a-4936-abd0-503a2bdcea45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e130d20bf7d34add86c1017682fa105d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:46.939695 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4acc23-344a-4936-abd0-503a2bdcea45 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:46.988441 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dc4acc23-344a-4936-abd0-503a2bdcea45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e130d20bf7d34add86c1017682fa105d', 'request_id': 'req-dc4acc23-344a-4936-abd0-503a2bdcea45'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:46.995185 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dc4acc23-344a-4936-abd0-503a2bdcea45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e130d20bf7d34add86c1017682fa105d', 'request_id': 'req-dc4acc23-344a-4936-abd0-503a2bdcea45'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:47.002262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dc4acc23-344a-4936-abd0-503a2bdcea45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e130d20bf7d34add86c1017682fa105d', 'request_id': 'req-dc4acc23-344a-4936-abd0-503a2bdcea45'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:47.013281 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dc4acc23-344a-4936-abd0-503a2bdcea45 admin admin] The token cache is being invalidate because project e130d20bf7d34add86c1017682fa105d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:07:47.013566 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dc4acc23-344a-4936-abd0-503a2bdcea45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:47.014205 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dc4acc23-344a-4936-abd0-503a2bdcea45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:47.015868 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1369/2724] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:07:46 2022] DELETE /identity/v3/projects/e130d20bf7d34add86c1017682fa105d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:47.021517 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86503d1c-9ea9-4899-bb55-50e67cd814e5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:47.022005 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86503d1c-9ea9-4899-bb55-50e67cd814e5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:47.022103 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86503d1c-9ea9-4899-bb55-50e67cd814e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:47.040840 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-86503d1c-9ea9-4899-bb55-50e67cd814e5 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:47.042072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-86503d1c-9ea9-4899-bb55-50e67cd814e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:47.103682 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1356/2725] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:47.113756 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87275967-8c97-4caa-97a6-d0a71f1ade1f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:47.114048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87275967-8c97-4caa-97a6-d0a71f1ade1f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:47.114351 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87275967-8c97-4caa-97a6-d0a71f1ade1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:47.134413 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-87275967-8c97-4caa-97a6-d0a71f1ade1f None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:47.135716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-87275967-8c97-4caa-97a6-d0a71f1ade1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:47.144302 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1370/2726] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:47.148787 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86503d1c-9ea9-4899-bb55-50e67cd814e5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:47.159599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86503d1c-9ea9-4899-bb55-50e67cd814e5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:47.164265 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750f9d2-c6b4-4e35-b7c2-be464f6566ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:47.164914 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750f9d2-c6b4-4e35-b7c2-be464f6566ec admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:47.165199 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750f9d2-c6b4-4e35-b7c2-be464f6566ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:47.165474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750f9d2-c6b4-4e35-b7c2-be464f6566ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:47.166192 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750f9d2-c6b4-4e35-b7c2-be464f6566ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:07:47.166503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750f9d2-c6b4-4e35-b7c2-be464f6566ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:47.168278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750f9d2-c6b4-4e35-b7c2-be464f6566ec admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:47.180444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1357/2727] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:07:47.185112 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87275967-8c97-4caa-97a6-d0a71f1ade1f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:47.194308 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87275967-8c97-4caa-97a6-d0a71f1ade1f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:47.199056 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3278fe-53bd-4554-ae29-7af0f9588a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:47.199985 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3278fe-53bd-4554-ae29-7af0f9588a75 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:47.200392 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3278fe-53bd-4554-ae29-7af0f9588a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:47.200677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3278fe-53bd-4554-ae29-7af0f9588a75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:47.201353 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3278fe-53bd-4554-ae29-7af0f9588a75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:47.203338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3278fe-53bd-4554-ae29-7af0f9588a75 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:47.229536 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1371/2728] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:07:47 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:07:47.234767 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750f9d2-c6b4-4e35-b7c2-be464f6566ec admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:47.244966 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750f9d2-c6b4-4e35-b7c2-be464f6566ec admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:47.295778 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:47.310123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99190c-b5ad-41a3-852d-15d4e4ddde40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:47.310792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99190c-b5ad-41a3-852d-15d4e4ddde40 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:47.311083 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99190c-b5ad-41a3-852d-15d4e4ddde40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:47.311362 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99190c-b5ad-41a3-852d-15d4e4ddde40 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:47.312092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99190c-b5ad-41a3-852d-15d4e4ddde40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:47.314265 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99190c-b5ad-41a3-852d-15d4e4ddde40 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:47.335252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1358/2729] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:07:47 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:07:47.339509 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3278fe-53bd-4554-ae29-7af0f9588a75 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:47.351614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3278fe-53bd-4554-ae29-7af0f9588a75 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:47.356565 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffd24a-839f-40c7-a5e7-ad8b9db225c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:47.357165 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffd24a-839f-40c7-a5e7-ad8b9db225c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:47.357450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffd24a-839f-40c7-a5e7-ad8b9db225c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:47.357726 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffd24a-839f-40c7-a5e7-ad8b9db225c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:47.358444 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ffd24a-839f-40c7-a5e7-ad8b9db225c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:47.360063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ffd24a-839f-40c7-a5e7-ad8b9db225c0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:47.370346 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1372/2730] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:47.374380 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99190c-b5ad-41a3-852d-15d4e4ddde40 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:47.384306 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99190c-b5ad-41a3-852d-15d4e4ddde40 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:47.390192 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ea0d1-6cab-4c3e-b675-be6cf712b1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:47.391864 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ea0d1-6cab-4c3e-b675-be6cf712b1a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:47.392156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ea0d1-6cab-4c3e-b675-be6cf712b1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:47.392439 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ea0d1-6cab-4c3e-b675-be6cf712b1a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:47.393182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ea0d1-6cab-4c3e-b675-be6cf712b1a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:47.394800 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ea0d1-6cab-4c3e-b675-be6cf712b1a6 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:47.406432 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1359/2731] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:47.410696 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffd24a-839f-40c7-a5e7-ad8b9db225c0 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:47.420835 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffd24a-839f-40c7-a5e7-ad8b9db225c0 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:47.425519 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9bd9f-cb65-4af2-9aaf-642465b43186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:47.426388 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9bd9f-cb65-4af2-9aaf-642465b43186 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:47.426711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9bd9f-cb65-4af2-9aaf-642465b43186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:47.426986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9bd9f-cb65-4af2-9aaf-642465b43186 admin admin] PATH_INFO: `/v3/projects/44aec8a89db742e6a5e0b71200e5f2ae/users/b1bb163351a54f0cadc61529878c02e6/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:47.441562 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9bd9f-cb65-4af2-9aaf-642465b43186 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44aec8a89db742e6a5e0b71200e5f2ae, user_id=b1bb163351a54f0cadc61529878c02e6, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:47.444316 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9bd9f-cb65-4af2-9aaf-642465b43186 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:47.457841 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1373/2732] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:07:47 2022] PUT /identity/v3/projects/44aec8a89db742e6a5e0b71200e5f2ae/users/b1bb163351a54f0cadc61529878c02e6/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:47.468863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b752ee6-3486-49c2-942d-0c0459acde3d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:47.469159 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b752ee6-3486-49c2-942d-0c0459acde3d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:47.469441 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b752ee6-3486-49c2-942d-0c0459acde3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:47.497070 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1b752ee6-3486-49c2-942d-0c0459acde3d None None] MFA Rules not processed for user `b1bb163351a54f0cadc61529878c02e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:47.498325 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1b752ee6-3486-49c2-942d-0c0459acde3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:47.566771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1360/2733] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:47.578711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9bd9f-cb65-4af2-9aaf-642465b43186 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:47.580540 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f8b9bd9f-cb65-4af2-9aaf-642465b43186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:47.645089 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9bd9f-cb65-4af2-9aaf-642465b43186 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:47.648827 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6cd08-c037-4ef1-afbc-77a2422fe94f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:47.649744 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6cd08-c037-4ef1-afbc-77a2422fe94f service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:47.650032 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6cd08-c037-4ef1-afbc-77a2422fe94f service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:47.650310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6cd08-c037-4ef1-afbc-77a2422fe94f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:47.658805 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6cd08-c037-4ef1-afbc-77a2422fe94f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:47.660230 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6cd08-c037-4ef1-afbc-77a2422fe94f service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:47.663726 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:47.671271 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1374/2734] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:47.690395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b752ee6-3486-49c2-942d-0c0459acde3d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:47.753406 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b752ee6-3486-49c2-942d-0c0459acde3d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:47.758080 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63729992-4e54-41ad-8708-01e4fb0e61c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:47.758997 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63729992-4e54-41ad-8708-01e4fb0e61c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:47.759275 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63729992-4e54-41ad-8708-01e4fb0e61c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:47.759546 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63729992-4e54-41ad-8708-01e4fb0e61c4 admin admin] PATH_INFO: `/v3/users/b1bb163351a54f0cadc61529878c02e6` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:47.760731 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63729992-4e54-41ad-8708-01e4fb0e61c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1bb163351a54f0cadc61529878c02e6)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:47.762844 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63729992-4e54-41ad-8708-01e4fb0e61c4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:47.811169 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-63729992-4e54-41ad-8708-01e4fb0e61c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1bb163351a54f0cadc61529878c02e6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:47.811847 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-63729992-4e54-41ad-8708-01e4fb0e61c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1bb163351a54f0cadc61529878c02e6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:47.819046 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-63729992-4e54-41ad-8708-01e4fb0e61c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1bb163351a54f0cadc61529878c02e6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:47.821017 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:47.834127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-63729992-4e54-41ad-8708-01e4fb0e61c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1bb163351a54f0cadc61529878c02e6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:47.834878 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-63729992-4e54-41ad-8708-01e4fb0e61c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1bb163351a54f0cadc61529878c02e6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:47.847919 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1361/2735] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:07:47 2022] DELETE /identity/v3/users/b1bb163351a54f0cadc61529878c02e6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:47.852386 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1a64b-8b77-4bef-9f5d-69fe7aa83733 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:47.852699 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1a64b-8b77-4bef-9f5d-69fe7aa83733 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:47.852980 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1a64b-8b77-4bef-9f5d-69fe7aa83733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:47.871119 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7cf1a64b-8b77-4bef-9f5d-69fe7aa83733 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:47.872557 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7cf1a64b-8b77-4bef-9f5d-69fe7aa83733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:47.933013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1375/2736] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:07:47.943050 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63729992-4e54-41ad-8708-01e4fb0e61c4 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:47.944538 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-63729992-4e54-41ad-8708-01e4fb0e61c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:48.008660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63729992-4e54-41ad-8708-01e4fb0e61c4 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:48.013184 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ad3f2-4343-4628-b4e7-c9ba830d1bfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:48.013866 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ad3f2-4343-4628-b4e7-c9ba830d1bfa service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:48.014188 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ad3f2-4343-4628-b4e7-c9ba830d1bfa service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:48.014472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ad3f2-4343-4628-b4e7-c9ba830d1bfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:48.023516 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ad3f2-4343-4628-b4e7-c9ba830d1bfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:48.025226 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ad3f2-4343-4628-b4e7-c9ba830d1bfa service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:48.035911 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1362/2737] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:48.401177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1a64b-8b77-4bef-9f5d-69fe7aa83733 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:48.403397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7cf1a64b-8b77-4bef-9f5d-69fe7aa83733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:48.415281 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1a64b-8b77-4bef-9f5d-69fe7aa83733 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:48.420430 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d0558-3b28-4db4-bb85-ee9d67fc1e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:48.421292 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d0558-3b28-4db4-bb85-ee9d67fc1e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:48.421578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d0558-3b28-4db4-bb85-ee9d67fc1e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:48.421854 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d0558-3b28-4db4-bb85-ee9d67fc1e56 admin admin] PATH_INFO: `/v3/projects/44aec8a89db742e6a5e0b71200e5f2ae` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:48.423093 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d0558-3b28-4db4-bb85-ee9d67fc1e56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44aec8a89db742e6a5e0b71200e5f2ae)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:48.425047 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d0558-3b28-4db4-bb85-ee9d67fc1e56 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:48.478711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d17d0558-3b28-4db4-bb85-ee9d67fc1e56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44aec8a89db742e6a5e0b71200e5f2ae', 'request_id': 'req-d17d0558-3b28-4db4-bb85-ee9d67fc1e56'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:48.501638 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d17d0558-3b28-4db4-bb85-ee9d67fc1e56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44aec8a89db742e6a5e0b71200e5f2ae', 'request_id': 'req-d17d0558-3b28-4db4-bb85-ee9d67fc1e56'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:48.517690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d17d0558-3b28-4db4-bb85-ee9d67fc1e56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44aec8a89db742e6a5e0b71200e5f2ae', 'request_id': 'req-d17d0558-3b28-4db4-bb85-ee9d67fc1e56'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:48.534885 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d17d0558-3b28-4db4-bb85-ee9d67fc1e56 admin admin] The token cache is being invalidate because project 44aec8a89db742e6a5e0b71200e5f2ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:07:48.535177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d17d0558-3b28-4db4-bb85-ee9d67fc1e56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:48.536384 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d17d0558-3b28-4db4-bb85-ee9d67fc1e56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:07:48.538006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1376/2738] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:07:48 2022] DELETE /identity/v3/projects/44aec8a89db742e6a5e0b71200e5f2ae => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:07:48.543174 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c8a25-ef4f-4a1c-aa22-527d72a22fd3 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:48.544250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c8a25-ef4f-4a1c-aa22-527d72a22fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:48.544746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c8a25-ef4f-4a1c-aa22-527d72a22fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:48.564059 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-697c8a25-ef4f-4a1c-aa22-527d72a22fd3 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:48.565123 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-697c8a25-ef4f-4a1c-aa22-527d72a22fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:48.626856 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1363/2739] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:48.634450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc001bb-0029-40c1-bc7e-3fb704bb9b30 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:48.634773 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc001bb-0029-40c1-bc7e-3fb704bb9b30 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:48.635059 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc001bb-0029-40c1-bc7e-3fb704bb9b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:48.656083 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0cc001bb-0029-40c1-bc7e-3fb704bb9b30 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:48.657301 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0cc001bb-0029-40c1-bc7e-3fb704bb9b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:48.671367 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c8a25-ef4f-4a1c-aa22-527d72a22fd3 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:48.677994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1377/2740] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:48.681851 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c8a25-ef4f-4a1c-aa22-527d72a22fd3 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:48.688517 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff44804-7cf9-4666-818c-8911eb4f8a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:48.689266 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff44804-7cf9-4666-818c-8911eb4f8a3a admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:48.689624 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff44804-7cf9-4666-818c-8911eb4f8a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:48.689971 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff44804-7cf9-4666-818c-8911eb4f8a3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:48.690865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff44804-7cf9-4666-818c-8911eb4f8a3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:07:48.691268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff44804-7cf9-4666-818c-8911eb4f8a3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:48.693004 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff44804-7cf9-4666-818c-8911eb4f8a3a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:48.707443 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1364/2741] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:07:48.711830 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc001bb-0029-40c1-bc7e-3fb704bb9b30 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:48.721420 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc001bb-0029-40c1-bc7e-3fb704bb9b30 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:48.726844 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79266913-3bc4-46a1-bae0-bf7d94c58ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:48.727594 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79266913-3bc4-46a1-bae0-bf7d94c58ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:48.728530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79266913-3bc4-46a1-bae0-bf7d94c58ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:48.728530 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79266913-3bc4-46a1-bae0-bf7d94c58ae1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:48.729035 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79266913-3bc4-46a1-bae0-bf7d94c58ae1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:48.731181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79266913-3bc4-46a1-bae0-bf7d94c58ae1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:48.766922 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1378/2742] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:07:48 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:07:48.767862 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff44804-7cf9-4666-818c-8911eb4f8a3a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:48.778695 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff44804-7cf9-4666-818c-8911eb4f8a3a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:48.841816 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d7633-2aa9-4cd3-991f-e786e4bbff10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:48.842669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d7633-2aa9-4cd3-991f-e786e4bbff10 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:48.843213 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d7633-2aa9-4cd3-991f-e786e4bbff10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:48.843582 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d7633-2aa9-4cd3-991f-e786e4bbff10 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:48.844348 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d7633-2aa9-4cd3-991f-e786e4bbff10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:48.846378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d7633-2aa9-4cd3-991f-e786e4bbff10 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:48.857457 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:48.872528 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1365/2743] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:07:48 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:07:48.876815 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79266913-3bc4-46a1-bae0-bf7d94c58ae1 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:48.888270 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79266913-3bc4-46a1-bae0-bf7d94c58ae1 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:48.893099 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd760eef-426d-45fd-925f-f650fca18f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:48.893867 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd760eef-426d-45fd-925f-f650fca18f4b admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:48.894233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd760eef-426d-45fd-925f-f650fca18f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:48.894589 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd760eef-426d-45fd-925f-f650fca18f4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:48.895538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd760eef-426d-45fd-925f-f650fca18f4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:48.897219 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd760eef-426d-45fd-925f-f650fca18f4b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:48.914617 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1379/2744] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:07:48.915734 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d7633-2aa9-4cd3-991f-e786e4bbff10 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:48.926249 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d7633-2aa9-4cd3-991f-e786e4bbff10 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:48.930965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bea82-e2e8-45d2-9adc-8719447a7347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:48.931637 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bea82-e2e8-45d2-9adc-8719447a7347 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:48.931922 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bea82-e2e8-45d2-9adc-8719447a7347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:48.932198 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bea82-e2e8-45d2-9adc-8719447a7347 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:48.932920 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741bea82-e2e8-45d2-9adc-8719447a7347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:48.934660 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741bea82-e2e8-45d2-9adc-8719447a7347 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:48.946233 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1366/2745] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:07:48.951809 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd760eef-426d-45fd-925f-f650fca18f4b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:48.963010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd760eef-426d-45fd-925f-f650fca18f4b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:48.968761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e27c1-5691-4a67-844c-2134225cf42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:48.969844 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e27c1-5691-4a67-844c-2134225cf42e admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:48.970126 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e27c1-5691-4a67-844c-2134225cf42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:48.970397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e27c1-5691-4a67-844c-2134225cf42e admin admin] PATH_INFO: `/v3/projects/8ee28f39049846188135a087b194d247/users/4fc47b0b30344863af004541d777915b/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:48.987247 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e27c1-5691-4a67-844c-2134225cf42e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ee28f39049846188135a087b194d247, user_id=4fc47b0b30344863af004541d777915b, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:48.989878 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e27c1-5691-4a67-844c-2134225cf42e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:49.009916 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1380/2746] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:07:48 2022] PUT /identity/v3/projects/8ee28f39049846188135a087b194d247/users/4fc47b0b30344863af004541d777915b/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:07:49.015807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af5af2-c097-41dc-ab90-9a9cdd4b30cd None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:49.016097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af5af2-c097-41dc-ab90-9a9cdd4b30cd None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:49.016375 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af5af2-c097-41dc-ab90-9a9cdd4b30cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:49.035937 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-38af5af2-c097-41dc-ab90-9a9cdd4b30cd None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:49.037136 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-38af5af2-c097-41dc-ab90-9a9cdd4b30cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:49.102138 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1367/2747] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:49.111833 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e27c1-5691-4a67-844c-2134225cf42e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:49.113584 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-296e27c1-5691-4a67-844c-2134225cf42e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:49.135087 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:49.177446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e27c1-5691-4a67-844c-2134225cf42e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:49.181070 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593418f-d480-46f1-9648-3d70e204e5de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:49.181715 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593418f-d480-46f1-9648-3d70e204e5de service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:49.181998 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593418f-d480-46f1-9648-3d70e204e5de service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:49.182273 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593418f-d480-46f1-9648-3d70e204e5de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:49.192503 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593418f-d480-46f1-9648-3d70e204e5de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:49.194042 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593418f-d480-46f1-9648-3d70e204e5de service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:49.204956 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1381/2748] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:50.244020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac88d-caa4-41c0-8427-d72be76234de None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:50.244524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac88d-caa4-41c0-8427-d72be76234de None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:50.244631 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac88d-caa4-41c0-8427-d72be76234de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:50.274088 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c9aac88d-caa4-41c0-8427-d72be76234de None None] MFA Rules not processed for user `4fc47b0b30344863af004541d777915b`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:50.275613 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c9aac88d-caa4-41c0-8427-d72be76234de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:50.344987 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1368/2749] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:50.356498 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593418f-d480-46f1-9648-3d70e204e5de service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:50.358088 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6593418f-d480-46f1-9648-3d70e204e5de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:50.418784 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593418f-d480-46f1-9648-3d70e204e5de service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:50.422439 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239d962-bb7c-4e15-89c2-5fa5702c1bd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:50.423121 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239d962-bb7c-4e15-89c2-5fa5702c1bd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:50.423405 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239d962-bb7c-4e15-89c2-5fa5702c1bd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:50.423703 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239d962-bb7c-4e15-89c2-5fa5702c1bd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:50.432426 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239d962-bb7c-4e15-89c2-5fa5702c1bd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:50.433898 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239d962-bb7c-4e15-89c2-5fa5702c1bd3 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:50.442975 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1382/2750] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:52.831852 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa42c3-98ab-4fad-80dd-550056064bc6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:52.832373 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa42c3-98ab-4fad-80dd-550056064bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:52.832529 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa42c3-98ab-4fad-80dd-550056064bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:52.850327 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-64fa42c3-98ab-4fad-80dd-550056064bc6 None None] MFA Rules not processed for user `4fc47b0b30344863af004541d777915b`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:52.851541 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-64fa42c3-98ab-4fad-80dd-550056064bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:52.859919 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1369/2751] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:52.965979 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524edcf9-47bd-4f40-a8f6-d5b1fbf7fc7e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:52.966683 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524edcf9-47bd-4f40-a8f6-d5b1fbf7fc7e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:52.966683 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524edcf9-47bd-4f40-a8f6-d5b1fbf7fc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:52.984223 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-524edcf9-47bd-4f40-a8f6-d5b1fbf7fc7e None None] MFA Rules not processed for user `922a970a740444a2959c6aa1d0921ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:52.985419 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-524edcf9-47bd-4f40-a8f6-d5b1fbf7fc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:53.042384 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1383/2752] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:07:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:08:02.845102 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=69 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:03.035341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=69 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:09.843567 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2be07-bd97-4fe7-8866-bfc573e6f778 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:09.844190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2be07-bd97-4fe7-8866-bfc573e6f778 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:09.844190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2be07-bd97-4fe7-8866-bfc573e6f778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:09.863191 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-25c2be07-bd97-4fe7-8866-bfc573e6f778 None None] MFA Rules not processed for user `4fc47b0b30344863af004541d777915b`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:08:09.864410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-25c2be07-bd97-4fe7-8866-bfc573e6f778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:09.872571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1370/2753] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:08:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:08:14.261920 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524edcf9-47bd-4f40-a8f6-d5b1fbf7fc7e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:14.273456 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524edcf9-47bd-4f40-a8f6-d5b1fbf7fc7e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:14.278016 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26dd65-249c-465d-b34a-f3ee6abdc0dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:14.278781 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26dd65-249c-465d-b34a-f3ee6abdc0dc service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:14.279065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26dd65-249c-465d-b34a-f3ee6abdc0dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:14.279343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26dd65-249c-465d-b34a-f3ee6abdc0dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:14.281133 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ea26dd65-249c-465d-b34a-f3ee6abdc0dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:14.347110 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26dd65-249c-465d-b34a-f3ee6abdc0dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:14.348552 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26dd65-249c-465d-b34a-f3ee6abdc0dc service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:14.359252 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1384/2754] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:08:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:08:19.857723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:24.353719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=30 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:31.369341 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07feb4-3749-4e1f-b043-8bd8c9226197 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:31.369861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07feb4-3749-4e1f-b043-8bd8c9226197 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:31.369969 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07feb4-3749-4e1f-b043-8bd8c9226197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:31.389182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-7f07feb4-3749-4e1f-b043-8bd8c9226197 None None] MFA Rules not processed for user `02c826fc0baa480393df4c884c9e35dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:08:31.390353 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7f07feb4-3749-4e1f-b043-8bd8c9226197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:31.445760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1371/2755] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Jun 16 19:08:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:08:31.492687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26dd65-249c-465d-b34a-f3ee6abdc0dc service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:31.494114 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ea26dd65-249c-465d-b34a-f3ee6abdc0dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:31.566013 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26dd65-249c-465d-b34a-f3ee6abdc0dc service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:31.571205 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b228bd-fba3-4496-90bb-d8f27c071cc2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b22fe0ab0354385924923d0d443acae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:31.571857 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b228bd-fba3-4496-90bb-d8f27c071cc2 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:31.572170 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b228bd-fba3-4496-90bb-d8f27c071cc2 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:31.572450 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b228bd-fba3-4496-90bb-d8f27c071cc2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:31.580774 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b228bd-fba3-4496-90bb-d8f27c071cc2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:31.582355 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b228bd-fba3-4496-90bb-d8f27c071cc2 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:31.591187 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1385/2756] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Jun 16 19:08:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:08:41.439047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=25 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:41.587223 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=32 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:46.721488 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5c044-7c2a-4c02-9be2-527536744dc5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:46.721989 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5c044-7c2a-4c02-9be2-527536744dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:46.722134 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5c044-7c2a-4c02-9be2-527536744dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:46.739990 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-a6b5c044-7c2a-4c02-9be2-527536744dc5 None None] MFA Rules not processed for user `02c826fc0baa480393df4c884c9e35dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:08:46.741186 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a6b5c044-7c2a-4c02-9be2-527536744dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:46.749205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1372/2757] 10.222.0.70 () {58 vars in 967 bytes} [Thu Jun 16 19:08:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:08:46.761343 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b228bd-fba3-4496-90bb-d8f27c071cc2 service swift] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:46.772018 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b228bd-fba3-4496-90bb-d8f27c071cc2 service swift] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:46.777414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131cf63-a743-49a8-baaf-b7ce25a23993 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b22fe0ab0354385924923d0d443acae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:46.778048 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131cf63-a743-49a8-baaf-b7ce25a23993 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:46.778336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131cf63-a743-49a8-baaf-b7ce25a23993 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:46.778651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131cf63-a743-49a8-baaf-b7ce25a23993 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:46.786799 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131cf63-a743-49a8-baaf-b7ce25a23993 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:46.788376 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131cf63-a743-49a8-baaf-b7ce25a23993 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:46.796992 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1386/2758] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Jun 16 19:08:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:08:48.715127 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5c044-7c2a-4c02-9be2-527536744dc5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:48.727126 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5c044-7c2a-4c02-9be2-527536744dc5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:48.732626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420f8c7-99db-4c3a-a9fd-23beae1e312c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:48.733481 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420f8c7-99db-4c3a-a9fd-23beae1e312c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:48.733750 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420f8c7-99db-4c3a-a9fd-23beae1e312c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:48.734091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420f8c7-99db-4c3a-a9fd-23beae1e312c admin admin] PATH_INFO: `/v3/users/4fc47b0b30344863af004541d777915b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:48.735228 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420f8c7-99db-4c3a-a9fd-23beae1e312c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fc47b0b30344863af004541d777915b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:48.738049 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420f8c7-99db-4c3a-a9fd-23beae1e312c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:48.786459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9420f8c7-99db-4c3a-a9fd-23beae1e312c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fc47b0b30344863af004541d777915b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:48.787136 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9420f8c7-99db-4c3a-a9fd-23beae1e312c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fc47b0b30344863af004541d777915b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:48.795756 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9420f8c7-99db-4c3a-a9fd-23beae1e312c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fc47b0b30344863af004541d777915b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:48.808647 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9420f8c7-99db-4c3a-a9fd-23beae1e312c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fc47b0b30344863af004541d777915b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:48.809399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9420f8c7-99db-4c3a-a9fd-23beae1e312c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fc47b0b30344863af004541d777915b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:48.823089 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1373/2759] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:08:48 2022] DELETE /identity/v3/users/4fc47b0b30344863af004541d777915b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:08:49.001977 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131cf63-a743-49a8-baaf-b7ce25a23993 service swift] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:49.013182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131cf63-a743-49a8-baaf-b7ce25a23993 service swift] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:49.074146 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5517be9-13b6-43af-a6df-8a9cbfb87a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:49.075044 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5517be9-13b6-43af-a6df-8a9cbfb87a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:49.075333 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5517be9-13b6-43af-a6df-8a9cbfb87a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:49.075640 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5517be9-13b6-43af-a6df-8a9cbfb87a05 admin admin] PATH_INFO: `/v3/projects/8ee28f39049846188135a087b194d247` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:49.076847 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5517be9-13b6-43af-a6df-8a9cbfb87a05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ee28f39049846188135a087b194d247)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:49.078807 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5517be9-13b6-43af-a6df-8a9cbfb87a05 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:49.127628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f5517be9-13b6-43af-a6df-8a9cbfb87a05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ee28f39049846188135a087b194d247', 'request_id': 'req-f5517be9-13b6-43af-a6df-8a9cbfb87a05'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:49.134875 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f5517be9-13b6-43af-a6df-8a9cbfb87a05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ee28f39049846188135a087b194d247', 'request_id': 'req-f5517be9-13b6-43af-a6df-8a9cbfb87a05'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:49.142094 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f5517be9-13b6-43af-a6df-8a9cbfb87a05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ee28f39049846188135a087b194d247', 'request_id': 'req-f5517be9-13b6-43af-a6df-8a9cbfb87a05'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:49.154035 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f5517be9-13b6-43af-a6df-8a9cbfb87a05 admin admin] The token cache is being invalidate because project 8ee28f39049846188135a087b194d247 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:08:49.154287 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f5517be9-13b6-43af-a6df-8a9cbfb87a05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:49.155036 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f5517be9-13b6-43af-a6df-8a9cbfb87a05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:49.156599 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1387/2760] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:08:49 2022] DELETE /identity/v3/projects/8ee28f39049846188135a087b194d247 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:08:54.379610 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66987f1-cb8d-4a79-9af5-753f8c372ee8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:54.380181 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66987f1-cb8d-4a79-9af5-753f8c372ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:54.380181 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66987f1-cb8d-4a79-9af5-753f8c372ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:54.398511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e66987f1-cb8d-4a79-9af5-753f8c372ee8 None None] MFA Rules not processed for user `55f5aba8355c436b94508156bca24ab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:08:54.399723 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e66987f1-cb8d-4a79-9af5-753f8c372ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:54.456048 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1374/2761] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:08:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:08:59.149338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:04.447865 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:44.389422 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5517be9-13b6-43af-a6df-8a9cbfb87a05 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:44.390843 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f5517be9-13b6-43af-a6df-8a9cbfb87a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:44.447482 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5517be9-13b6-43af-a6df-8a9cbfb87a05 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:44.451081 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c313b2-7e9d-4f60-ae96-746ab57b8ec0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:44.451715 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c313b2-7e9d-4f60-ae96-746ab57b8ec0 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:44.451995 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c313b2-7e9d-4f60-ae96-746ab57b8ec0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:44.452269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c313b2-7e9d-4f60-ae96-746ab57b8ec0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:44.454049 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-65c313b2-7e9d-4f60-ae96-746ab57b8ec0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:44.523860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c313b2-7e9d-4f60-ae96-746ab57b8ec0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:44.525290 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c313b2-7e9d-4f60-ae96-746ab57b8ec0 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:44.536656 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1388/2762] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:09:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:09:45.948888 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66987f1-cb8d-4a79-9af5-753f8c372ee8 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:45.950431 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e66987f1-cb8d-4a79-9af5-753f8c372ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:46.009456 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66987f1-cb8d-4a79-9af5-753f8c372ee8 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:46.012977 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ffb4a-5e05-4271-9325-d69b1522a9bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:46.013740 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ffb4a-5e05-4271-9325-d69b1522a9bc service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:46.014024 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ffb4a-5e05-4271-9325-d69b1522a9bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:46.014301 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ffb4a-5e05-4271-9325-d69b1522a9bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:46.016116 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9f4ffb4a-5e05-4271-9325-d69b1522a9bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:46.030635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ffb4a-5e05-4271-9325-d69b1522a9bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:46.032058 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ffb4a-5e05-4271-9325-d69b1522a9bc service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:46.044102 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1375/2763] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:09:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:09:53.178532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c313b2-7e9d-4f60-ae96-746ab57b8ec0 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:53.180370 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-65c313b2-7e9d-4f60-ae96-746ab57b8ec0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:53.195621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c313b2-7e9d-4f60-ae96-746ab57b8ec0 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:53.203769 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b48ec-9ff0-4dcd-a052-36cd37ee7eeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:53.204646 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b48ec-9ff0-4dcd-a052-36cd37ee7eeb service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:53.205041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b48ec-9ff0-4dcd-a052-36cd37ee7eeb service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:53.205472 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b48ec-9ff0-4dcd-a052-36cd37ee7eeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:53.208908 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bd5b48ec-9ff0-4dcd-a052-36cd37ee7eeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:53.224829 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b48ec-9ff0-4dcd-a052-36cd37ee7eeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:53.226482 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b48ec-9ff0-4dcd-a052-36cd37ee7eeb service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:53.239301 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1389/2764] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:09:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:09:56.038971 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=26 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:56.262675 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678970a-4c47-4f5d-8046-b03801eefb08 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:56.262972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678970a-4c47-4f5d-8046-b03801eefb08 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:56.263253 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678970a-4c47-4f5d-8046-b03801eefb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:56.281463 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1678970a-4c47-4f5d-8046-b03801eefb08 None None] MFA Rules not processed for user `55f5aba8355c436b94508156bca24ab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:56.282691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1678970a-4c47-4f5d-8046-b03801eefb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:56.290010 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1376/2765] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:09:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:09:56.922097 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b48ec-9ff0-4dcd-a052-36cd37ee7eeb service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:56.923551 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bd5b48ec-9ff0-4dcd-a052-36cd37ee7eeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:56.996475 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b48ec-9ff0-4dcd-a052-36cd37ee7eeb service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:57.001566 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df717f-6ebf-4740-bb17-fb5722e5649e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '880d0299c38c4eb897c669a512e599be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:57.002216 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df717f-6ebf-4740-bb17-fb5722e5649e service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:57.002503 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df717f-6ebf-4740-bb17-fb5722e5649e service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:57.002821 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df717f-6ebf-4740-bb17-fb5722e5649e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:57.004602 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-95df717f-6ebf-4740-bb17-fb5722e5649e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:57.016372 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df717f-6ebf-4740-bb17-fb5722e5649e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:57.017974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df717f-6ebf-4740-bb17-fb5722e5649e service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:57.028766 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1390/2766] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:09:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:09:59.315792 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678970a-4c47-4f5d-8046-b03801eefb08 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:59.338445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678970a-4c47-4f5d-8046-b03801eefb08 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:59.344412 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba4da2-5216-4f47-af5f-93890f1ee345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:59.345453 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba4da2-5216-4f47-af5f-93890f1ee345 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:59.346000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba4da2-5216-4f47-af5f-93890f1ee345 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:59.346511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba4da2-5216-4f47-af5f-93890f1ee345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:59.348751 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-21ba4da2-5216-4f47-af5f-93890f1ee345 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:59.371581 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba4da2-5216-4f47-af5f-93890f1ee345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:59.374052 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba4da2-5216-4f47-af5f-93890f1ee345 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:59.386958 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1377/2767] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:09:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:10:07.024329 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:10:08.963020 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df717f-6ebf-4740-bb17-fb5722e5649e service placement] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:10:08.964623 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-95df717f-6ebf-4740-bb17-fb5722e5649e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:10:08.982342 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df717f-6ebf-4740-bb17-fb5722e5649e service placement] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:10:08.987942 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f196542-4416-437d-a429-940ebb9c7df4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:10:08.988708 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f196542-4416-437d-a429-940ebb9c7df4 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:10:08.988996 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f196542-4416-437d-a429-940ebb9c7df4 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:10:08.989293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f196542-4416-437d-a429-940ebb9c7df4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:10:08.991272 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9f196542-4416-437d-a429-940ebb9c7df4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:10:09.003716 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f196542-4416-437d-a429-940ebb9c7df4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:10:09.005411 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f196542-4416-437d-a429-940ebb9c7df4 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:10:09.016739 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1391/2768] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:10:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:10:09.382264 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=9 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:10:11.509114 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba4da2-5216-4f47-af5f-93890f1ee345 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:10:11.518487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba4da2-5216-4f47-af5f-93890f1ee345 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:10:11.522549 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78482f-e20a-4492-b420-0d61203cc2cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:10:11.523244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78482f-e20a-4492-b420-0d61203cc2cc service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:10:11.523530 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78482f-e20a-4492-b420-0d61203cc2cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:10:11.523810 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78482f-e20a-4492-b420-0d61203cc2cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:10:11.525702 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9b78482f-e20a-4492-b420-0d61203cc2cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:10:11.595900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78482f-e20a-4492-b420-0d61203cc2cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:10:11.597874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78482f-e20a-4492-b420-0d61203cc2cc service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:10:11.610042 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1378/2769] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:10:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:10:18.493525 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f196542-4416-437d-a429-940ebb9c7df4 service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:10:18.495064 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9f196542-4416-437d-a429-940ebb9c7df4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:10:18.511229 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f196542-4416-437d-a429-940ebb9c7df4 service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:10:18.515884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e522ab-b6a0-419d-b736-0b4429c18732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:10:18.516750 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e522ab-b6a0-419d-b736-0b4429c18732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:10:18.517032 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e522ab-b6a0-419d-b736-0b4429c18732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:10:18.517305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e522ab-b6a0-419d-b736-0b4429c18732 admin admin] PATH_INFO: `/v3/users/922a970a740444a2959c6aa1d0921ee7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:10:18.518481 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e522ab-b6a0-419d-b736-0b4429c18732 admin admin] RBAC: Authorizing `identity:delete_user(user_id=922a970a740444a2959c6aa1d0921ee7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:10:18.520412 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e522ab-b6a0-419d-b736-0b4429c18732 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:10:18.570660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-41e522ab-b6a0-419d-b736-0b4429c18732 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '922a970a740444a2959c6aa1d0921ee7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:18.571365 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-41e522ab-b6a0-419d-b736-0b4429c18732 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '922a970a740444a2959c6aa1d0921ee7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:18.572440 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-41e522ab-b6a0-419d-b736-0b4429c18732 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '922a970a740444a2959c6aa1d0921ee7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:18.579120 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-41e522ab-b6a0-419d-b736-0b4429c18732 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '922a970a740444a2959c6aa1d0921ee7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:18.586461 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-41e522ab-b6a0-419d-b736-0b4429c18732 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '922a970a740444a2959c6aa1d0921ee7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:18.603859 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1392/2770] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:10:18 2022] DELETE /identity/v3/users/922a970a740444a2959c6aa1d0921ee7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:10:18.780894 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78482f-e20a-4492-b420-0d61203cc2cc service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:10:18.792222 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78482f-e20a-4492-b420-0d61203cc2cc service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:10:18.859890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dac86f-2d47-4faa-980d-d064c2c61e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:10:18.860754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dac86f-2d47-4faa-980d-d064c2c61e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:10:18.861040 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dac86f-2d47-4faa-980d-d064c2c61e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:10:18.861318 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dac86f-2d47-4faa-980d-d064c2c61e7c admin admin] PATH_INFO: `/v3/projects/0e52189be8d24c9894ef05cc7acf367c` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:10:18.862500 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dac86f-2d47-4faa-980d-d064c2c61e7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e52189be8d24c9894ef05cc7acf367c)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:10:18.864965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dac86f-2d47-4faa-980d-d064c2c61e7c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:10:18.911756 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-91dac86f-2d47-4faa-980d-d064c2c61e7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e52189be8d24c9894ef05cc7acf367c', 'request_id': 'req-91dac86f-2d47-4faa-980d-d064c2c61e7c'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:18.919649 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-91dac86f-2d47-4faa-980d-d064c2c61e7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e52189be8d24c9894ef05cc7acf367c', 'request_id': 'req-91dac86f-2d47-4faa-980d-d064c2c61e7c'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:18.925903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-91dac86f-2d47-4faa-980d-d064c2c61e7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e52189be8d24c9894ef05cc7acf367c', 'request_id': 'req-91dac86f-2d47-4faa-980d-d064c2c61e7c'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:18.936979 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-91dac86f-2d47-4faa-980d-d064c2c61e7c admin admin] The token cache is being invalidate because project 0e52189be8d24c9894ef05cc7acf367c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:10:18.937271 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-91dac86f-2d47-4faa-980d-d064c2c61e7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:18.937933 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-91dac86f-2d47-4faa-980d-d064c2c61e7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:18.939511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1379/2771] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:10:18 2022] DELETE /identity/v3/projects/0e52189be8d24c9894ef05cc7acf367c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:10:18.943793 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e522ab-b6a0-419d-b736-0b4429c18732 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:10:18.945231 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-41e522ab-b6a0-419d-b736-0b4429c18732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:10:18.958946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e522ab-b6a0-419d-b736-0b4429c18732 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:10:19.017331 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5bb59-eb59-43a7-9050-c21f4363aa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:10:19.018249 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5bb59-eb59-43a7-9050-c21f4363aa50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:10:19.018542 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5bb59-eb59-43a7-9050-c21f4363aa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:10:19.018978 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5bb59-eb59-43a7-9050-c21f4363aa50 admin admin] PATH_INFO: `/v3/users/9af6d57157e74a83a9776604dfd86f50` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:10:19.020224 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5bb59-eb59-43a7-9050-c21f4363aa50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af6d57157e74a83a9776604dfd86f50)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:10:19.022235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5bb59-eb59-43a7-9050-c21f4363aa50 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:10:19.068660 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-fea5bb59-eb59-43a7-9050-c21f4363aa50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af6d57157e74a83a9776604dfd86f50'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:19.069306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-fea5bb59-eb59-43a7-9050-c21f4363aa50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af6d57157e74a83a9776604dfd86f50'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:19.069909 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-fea5bb59-eb59-43a7-9050-c21f4363aa50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af6d57157e74a83a9776604dfd86f50'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:19.077336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-fea5bb59-eb59-43a7-9050-c21f4363aa50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af6d57157e74a83a9776604dfd86f50'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:19.082770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:10:19.085535 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-fea5bb59-eb59-43a7-9050-c21f4363aa50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af6d57157e74a83a9776604dfd86f50'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:19.104897 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1393/2772] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:10:18 2022] DELETE /identity/v3/users/9af6d57157e74a83a9776604dfd86f50 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:10:19.282347 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dac86f-2d47-4faa-980d-d064c2c61e7c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:10:19.284047 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-91dac86f-2d47-4faa-980d-d064c2c61e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:10:19.299742 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dac86f-2d47-4faa-980d-d064c2c61e7c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:10:19.321806 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:10:19.363948 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4797197-90c1-464a-82c7-e5ceb4ae12b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:10:19.365134 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4797197-90c1-464a-82c7-e5ceb4ae12b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:10:19.365134 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4797197-90c1-464a-82c7-e5ceb4ae12b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:10:19.365478 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4797197-90c1-464a-82c7-e5ceb4ae12b6 admin admin] PATH_INFO: `/v3/projects/210593d00452434e9eb7568a4084fc0b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:10:19.366552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4797197-90c1-464a-82c7-e5ceb4ae12b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=210593d00452434e9eb7568a4084fc0b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:10:19.369117 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4797197-90c1-464a-82c7-e5ceb4ae12b6 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:10:19.421954 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a4797197-90c1-464a-82c7-e5ceb4ae12b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '210593d00452434e9eb7568a4084fc0b', 'request_id': 'req-a4797197-90c1-464a-82c7-e5ceb4ae12b6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:19.430457 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a4797197-90c1-464a-82c7-e5ceb4ae12b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '210593d00452434e9eb7568a4084fc0b', 'request_id': 'req-a4797197-90c1-464a-82c7-e5ceb4ae12b6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:19.436856 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a4797197-90c1-464a-82c7-e5ceb4ae12b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '210593d00452434e9eb7568a4084fc0b', 'request_id': 'req-a4797197-90c1-464a-82c7-e5ceb4ae12b6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:19.447775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a4797197-90c1-464a-82c7-e5ceb4ae12b6 admin admin] The token cache is being invalidate because project 210593d00452434e9eb7568a4084fc0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:10:19.448216 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a4797197-90c1-464a-82c7-e5ceb4ae12b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:19.448855 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a4797197-90c1-464a-82c7-e5ceb4ae12b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:10:19.450400 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1380/2773] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:10:19 2022] DELETE /identity/v3/projects/210593d00452434e9eb7568a4084fc0b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:10:29.101269 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:10:29.448334 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:11:06.812262 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbbf5e-f2f1-4058-a00c-5a2a67e10b44 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:11:06.813145 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbbf5e-f2f1-4058-a00c-5a2a67e10b44 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:11:06.813329 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbbf5e-f2f1-4058-a00c-5a2a67e10b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:11:06.832023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-69bbbf5e-f2f1-4058-a00c-5a2a67e10b44 None None] MFA Rules not processed for user `55f5aba8355c436b94508156bca24ab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:11:06.833236 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-69bbbf5e-f2f1-4058-a00c-5a2a67e10b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:11:06.891895 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1394/2774] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:11:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:11:07.216680 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4797197-90c1-464a-82c7-e5ceb4ae12b6 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:11:07.218136 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a4797197-90c1-464a-82c7-e5ceb4ae12b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:11:07.286472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4797197-90c1-464a-82c7-e5ceb4ae12b6 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:11:07.290895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c945dc-74d9-4176-ac45-cc5da10eb62b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:11:07.291533 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c945dc-74d9-4176-ac45-cc5da10eb62b service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:11:07.291823 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c945dc-74d9-4176-ac45-cc5da10eb62b service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:11:07.292100 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c945dc-74d9-4176-ac45-cc5da10eb62b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:11:07.293903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a1c945dc-74d9-4176-ac45-cc5da10eb62b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:11:07.304090 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c945dc-74d9-4176-ac45-cc5da10eb62b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:11:07.305550 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c945dc-74d9-4176-ac45-cc5da10eb62b service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:11:07.313838 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1381/2775] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:11:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:11:16.884984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=25 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:11:17.310070 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=26 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:11:19.808121 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbbf5e-f2f1-4058-a00c-5a2a67e10b44 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:11:19.810206 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-69bbbf5e-f2f1-4058-a00c-5a2a67e10b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:11:19.981826 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbbf5e-f2f1-4058-a00c-5a2a67e10b44 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:11:19.990546 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37988a89-b24d-4d5f-a2c1-2e6e52e1d0c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8342da838fe4462b9de2817ad3ac4ff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:11:19.991595 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37988a89-b24d-4d5f-a2c1-2e6e52e1d0c2 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:11:19.992183 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37988a89-b24d-4d5f-a2c1-2e6e52e1d0c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:11:19.992619 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37988a89-b24d-4d5f-a2c1-2e6e52e1d0c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:11:19.995482 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-37988a89-b24d-4d5f-a2c1-2e6e52e1d0c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:11:20.013963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37988a89-b24d-4d5f-a2c1-2e6e52e1d0c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:11:20.016447 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37988a89-b24d-4d5f-a2c1-2e6e52e1d0c2 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:11:20.033463 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1395/2776] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:11:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:11:30.026177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=56 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:17.563960 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c945dc-74d9-4176-ac45-cc5da10eb62b service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:17.566216 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a1c945dc-74d9-4176-ac45-cc5da10eb62b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:17.623273 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37988a89-b24d-4d5f-a2c1-2e6e52e1d0c2 service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:17.626167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-37988a89-b24d-4d5f-a2c1-2e6e52e1d0c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:17.709182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c945dc-74d9-4176-ac45-cc5da10eb62b service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:17.715329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb916e-f7c6-45f1-9f22-0d2c639e8481 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '880d0299c38c4eb897c669a512e599be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:17.715967 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb916e-f7c6-45f1-9f22-0d2c639e8481 service placement] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:17.716246 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb916e-f7c6-45f1-9f22-0d2c639e8481 service placement] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:17.716524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb916e-f7c6-45f1-9f22-0d2c639e8481 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:17.718286 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-14bb916e-f7c6-45f1-9f22-0d2c639e8481 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:17.730522 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb916e-f7c6-45f1-9f22-0d2c639e8481 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:17.732156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb916e-f7c6-45f1-9f22-0d2c639e8481 service placement] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:17.733532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37988a89-b24d-4d5f-a2c1-2e6e52e1d0c2 service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:17.738336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8ab0f-6375-4db3-a40c-e46a4c9e9e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:17.739237 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8ab0f-6375-4db3-a40c-e46a4c9e9e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:17.739522 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8ab0f-6375-4db3-a40c-e46a4c9e9e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:17.739822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8ab0f-6375-4db3-a40c-e46a4c9e9e1f admin admin] PATH_INFO: `/v3/users/55f5aba8355c436b94508156bca24ab3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:17.741010 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8ab0f-6375-4db3-a40c-e46a4c9e9e1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=55f5aba8355c436b94508156bca24ab3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:17.741969 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1382/2777] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:12:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:12:17.743633 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8ab0f-6375-4db3-a40c-e46a4c9e9e1f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:17.809360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f0b8ab0f-6375-4db3-a40c-e46a4c9e9e1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55f5aba8355c436b94508156bca24ab3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:17.810009 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f0b8ab0f-6375-4db3-a40c-e46a4c9e9e1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55f5aba8355c436b94508156bca24ab3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:17.810638 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f0b8ab0f-6375-4db3-a40c-e46a4c9e9e1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55f5aba8355c436b94508156bca24ab3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:17.818250 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f0b8ab0f-6375-4db3-a40c-e46a4c9e9e1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55f5aba8355c436b94508156bca24ab3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:17.826232 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f0b8ab0f-6375-4db3-a40c-e46a4c9e9e1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55f5aba8355c436b94508156bca24ab3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:17.843439 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1396/2778] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:12:17 2022] DELETE /identity/v3/users/55f5aba8355c436b94508156bca24ab3 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:12:18.019484 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb916e-f7c6-45f1-9f22-0d2c639e8481 service placement] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:18.033663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb916e-f7c6-45f1-9f22-0d2c639e8481 service placement] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:18.092827 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b123b2d-c8d1-4b05-bc32-ec64b95d6922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:18.093709 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b123b2d-c8d1-4b05-bc32-ec64b95d6922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:18.093989 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b123b2d-c8d1-4b05-bc32-ec64b95d6922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:18.094263 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b123b2d-c8d1-4b05-bc32-ec64b95d6922 admin admin] PATH_INFO: `/v3/projects/bf39ecb69ea7474d940ad17f8bab4874` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:18.095474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b123b2d-c8d1-4b05-bc32-ec64b95d6922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf39ecb69ea7474d940ad17f8bab4874)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:18.097399 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b123b2d-c8d1-4b05-bc32-ec64b95d6922 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:18.141673 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:18.145878 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9b123b2d-c8d1-4b05-bc32-ec64b95d6922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf39ecb69ea7474d940ad17f8bab4874', 'request_id': 'req-9b123b2d-c8d1-4b05-bc32-ec64b95d6922'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:18.154329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9b123b2d-c8d1-4b05-bc32-ec64b95d6922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf39ecb69ea7474d940ad17f8bab4874', 'request_id': 'req-9b123b2d-c8d1-4b05-bc32-ec64b95d6922'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:18.160618 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9b123b2d-c8d1-4b05-bc32-ec64b95d6922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf39ecb69ea7474d940ad17f8bab4874', 'request_id': 'req-9b123b2d-c8d1-4b05-bc32-ec64b95d6922'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:18.171725 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9b123b2d-c8d1-4b05-bc32-ec64b95d6922 admin admin] The token cache is being invalidate because project bf39ecb69ea7474d940ad17f8bab4874 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:12:18.172011 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9b123b2d-c8d1-4b05-bc32-ec64b95d6922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:18.172634 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9b123b2d-c8d1-4b05-bc32-ec64b95d6922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:18.174179 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1383/2779] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:12:18 2022] DELETE /identity/v3/projects/bf39ecb69ea7474d940ad17f8bab4874 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:18.179023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32537071-6d60-4b53-8b8f-bfd72072e747 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:18.179309 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32537071-6d60-4b53-8b8f-bfd72072e747 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:18.179584 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32537071-6d60-4b53-8b8f-bfd72072e747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:18.236080 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-32537071-6d60-4b53-8b8f-bfd72072e747 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:18.237273 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-32537071-6d60-4b53-8b8f-bfd72072e747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:18.250319 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:18.300421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1397/2780] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:18.309078 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c38a2-7524-4143-b145-b5b3f4fda522 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:18.309370 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c38a2-7524-4143-b145-b5b3f4fda522 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:18.309652 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c38a2-7524-4143-b145-b5b3f4fda522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:18.327868 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-334c38a2-7524-4143-b145-b5b3f4fda522 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:18.329053 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-334c38a2-7524-4143-b145-b5b3f4fda522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:18.337532 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1384/2781] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:18.341834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32537071-6d60-4b53-8b8f-bfd72072e747 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:18.351463 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32537071-6d60-4b53-8b8f-bfd72072e747 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:18.356808 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9b427-31a7-401a-85cc-469976b64db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:18.357458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9b427-31a7-401a-85cc-469976b64db3 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:18.357737 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9b427-31a7-401a-85cc-469976b64db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:18.358008 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9b427-31a7-401a-85cc-469976b64db3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:18.358761 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9b427-31a7-401a-85cc-469976b64db3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:12:18.359066 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9b427-31a7-401a-85cc-469976b64db3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:18.360686 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9b427-31a7-401a-85cc-469976b64db3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:18.373541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1398/2782] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:12:18.377843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c38a2-7524-4143-b145-b5b3f4fda522 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:18.387560 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c38a2-7524-4143-b145-b5b3f4fda522 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:18.392309 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dce3a-01a6-4357-b595-218a57c8eeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:18.392950 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dce3a-01a6-4357-b595-218a57c8eeb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:18.393235 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dce3a-01a6-4357-b595-218a57c8eeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:18.393511 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dce3a-01a6-4357-b595-218a57c8eeb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:18.394190 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dce3a-01a6-4357-b595-218a57c8eeb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:18.396182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dce3a-01a6-4357-b595-218a57c8eeb1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:18.430287 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1385/2783] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:12:18 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:12:18.435066 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9b427-31a7-401a-85cc-469976b64db3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:18.445628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9b427-31a7-401a-85cc-469976b64db3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:18.513676 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c5871-c2aa-48fd-a437-ea07d7c75f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:18.514381 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c5871-c2aa-48fd-a437-ea07d7c75f01 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:18.514684 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c5871-c2aa-48fd-a437-ea07d7c75f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:18.514969 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c5871-c2aa-48fd-a437-ea07d7c75f01 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:18.515674 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c5871-c2aa-48fd-a437-ea07d7c75f01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:18.517604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c5871-c2aa-48fd-a437-ea07d7c75f01 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:18.539260 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1399/2784] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:12:18 2022] POST /identity/v3/users => generated 283 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:12:18.544884 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dce3a-01a6-4357-b595-218a57c8eeb1 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:18.556627 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dce3a-01a6-4357-b595-218a57c8eeb1 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:18.561497 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8a938-00c0-49e0-a1f1-9eb9e6b76c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:18.562188 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8a938-00c0-49e0-a1f1-9eb9e6b76c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:18.562447 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8a938-00c0-49e0-a1f1-9eb9e6b76c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:18.562755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8a938-00c0-49e0-a1f1-9eb9e6b76c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:18.563479 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8a938-00c0-49e0-a1f1-9eb9e6b76c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:18.565072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8a938-00c0-49e0-a1f1-9eb9e6b76c1d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:18.576482 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1386/2785] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:12:18.580517 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c5871-c2aa-48fd-a437-ea07d7c75f01 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:18.591298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c5871-c2aa-48fd-a437-ea07d7c75f01 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:18.596326 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02695b3-7174-4d07-b7b0-0ff3a879f553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:18.597064 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02695b3-7174-4d07-b7b0-0ff3a879f553 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:18.597349 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02695b3-7174-4d07-b7b0-0ff3a879f553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:18.597628 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02695b3-7174-4d07-b7b0-0ff3a879f553 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:18.598369 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02695b3-7174-4d07-b7b0-0ff3a879f553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:18.600073 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02695b3-7174-4d07-b7b0-0ff3a879f553 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:18.612470 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1400/2786] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:12:18.616776 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8a938-00c0-49e0-a1f1-9eb9e6b76c1d admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:18.628698 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8a938-00c0-49e0-a1f1-9eb9e6b76c1d admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:18.633689 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6f9eb-5975-4579-bf6b-13cb36b60fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:18.634626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6f9eb-5975-4579-bf6b-13cb36b60fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:18.635167 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6f9eb-5975-4579-bf6b-13cb36b60fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:18.635600 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6f9eb-5975-4579-bf6b-13cb36b60fc7 admin admin] PATH_INFO: `/v3/projects/f2295df09b8c4f8c9e5bcf569d783b1b/users/4c9097fd72574b07bd22e6ab26b56f32/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:18.651841 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e6f9eb-5975-4579-bf6b-13cb36b60fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2295df09b8c4f8c9e5bcf569d783b1b, user_id=4c9097fd72574b07bd22e6ab26b56f32, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:18.654744 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e6f9eb-5975-4579-bf6b-13cb36b60fc7 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:18.670799 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1387/2787] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:12:18 2022] PUT /identity/v3/projects/f2295df09b8c4f8c9e5bcf569d783b1b/users/4c9097fd72574b07bd22e6ab26b56f32/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:18.681858 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7582b-c527-407c-84a0-be51ebf6aa55 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:18.682399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7582b-c527-407c-84a0-be51ebf6aa55 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:18.682399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7582b-c527-407c-84a0-be51ebf6aa55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:18.712077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8df7582b-c527-407c-84a0-be51ebf6aa55 None None] MFA Rules not processed for user `4c9097fd72574b07bd22e6ab26b56f32`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:18.713298 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8df7582b-c527-407c-84a0-be51ebf6aa55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:18.779912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1401/2788] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:18.784545 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6f9eb-5975-4579-bf6b-13cb36b60fc7 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:18.846973 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6f9eb-5975-4579-bf6b-13cb36b60fc7 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:18.851961 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02a14c-5e79-4820-bdb6-a166ff337cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:18.852705 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02a14c-5e79-4820-bdb6-a166ff337cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:18.852981 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02a14c-5e79-4820-bdb6-a166ff337cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:18.853252 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02a14c-5e79-4820-bdb6-a166ff337cc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:18.853931 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02a14c-5e79-4820-bdb6-a166ff337cc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:18.855849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02a14c-5e79-4820-bdb6-a166ff337cc3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:18.881774 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1388/2789] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:12:18 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:12:18.887720 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7582b-c527-407c-84a0-be51ebf6aa55 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:18.898708 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7582b-c527-407c-84a0-be51ebf6aa55 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:18.911944 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:18.963895 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d33788-07cd-44eb-a8b7-6c2602cdd982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:18.964587 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d33788-07cd-44eb-a8b7-6c2602cdd982 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:18.964870 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d33788-07cd-44eb-a8b7-6c2602cdd982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:18.965148 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d33788-07cd-44eb-a8b7-6c2602cdd982 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:18.965842 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d33788-07cd-44eb-a8b7-6c2602cdd982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:18.967811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d33788-07cd-44eb-a8b7-6c2602cdd982 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:18.994052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1402/2790] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:12:18 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:12:18.995537 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02a14c-5e79-4820-bdb6-a166ff337cc3 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:19.008861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02a14c-5e79-4820-bdb6-a166ff337cc3 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:19.014065 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdebfc-1e59-4304-bb69-84221cd11a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:19.014903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdebfc-1e59-4304-bb69-84221cd11a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:19.015235 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdebfc-1e59-4304-bb69-84221cd11a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:19.015553 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdebfc-1e59-4304-bb69-84221cd11a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:19.016386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdebfc-1e59-4304-bb69-84221cd11a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:19.018044 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdebfc-1e59-4304-bb69-84221cd11a20 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:19.030892 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1389/2791] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:12:19.034926 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d33788-07cd-44eb-a8b7-6c2602cdd982 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:19.045549 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d33788-07cd-44eb-a8b7-6c2602cdd982 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:19.050237 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049eb893-a9a1-4e72-9576-f2b08f793541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:19.051173 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049eb893-a9a1-4e72-9576-f2b08f793541 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:19.051453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049eb893-a9a1-4e72-9576-f2b08f793541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:19.051728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049eb893-a9a1-4e72-9576-f2b08f793541 admin admin] PATH_INFO: `/v3/projects/17fc908900ca4ca5bc551b4b89708a0e/users/4a656a11adc346d8844efcd564599f3b/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:19.067470 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049eb893-a9a1-4e72-9576-f2b08f793541 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17fc908900ca4ca5bc551b4b89708a0e, user_id=4a656a11adc346d8844efcd564599f3b, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:19.070144 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049eb893-a9a1-4e72-9576-f2b08f793541 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:19.085541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1403/2792] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:12:19 2022] PUT /identity/v3/projects/17fc908900ca4ca5bc551b4b89708a0e/users/4a656a11adc346d8844efcd564599f3b/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:19.090908 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdebfc-1e59-4304-bb69-84221cd11a20 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:19.102071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdebfc-1e59-4304-bb69-84221cd11a20 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:19.146355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:19.171451 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082952a-fdae-4980-928d-5b3f3a2beff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:19.172028 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082952a-fdae-4980-928d-5b3f3a2beff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:19.172323 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082952a-fdae-4980-928d-5b3f3a2beff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:19.172580 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082952a-fdae-4980-928d-5b3f3a2beff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:19.173302 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082952a-fdae-4980-928d-5b3f3a2beff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:19.174943 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082952a-fdae-4980-928d-5b3f3a2beff8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:19.186269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1390/2793] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:12:19.190477 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049eb893-a9a1-4e72-9576-f2b08f793541 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:19.202171 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049eb893-a9a1-4e72-9576-f2b08f793541 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:19.207063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23d7ef-b104-418b-a73c-f86dd06bc4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:19.207963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23d7ef-b104-418b-a73c-f86dd06bc4ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:19.208246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23d7ef-b104-418b-a73c-f86dd06bc4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:19.208517 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23d7ef-b104-418b-a73c-f86dd06bc4ff admin admin] PATH_INFO: `/v3/projects/17fc908900ca4ca5bc551b4b89708a0e/users/4a656a11adc346d8844efcd564599f3b/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:19.210500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23d7ef-b104-418b-a73c-f86dd06bc4ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=17fc908900ca4ca5bc551b4b89708a0e, user_id=4a656a11adc346d8844efcd564599f3b, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:19.213442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23d7ef-b104-418b-a73c-f86dd06bc4ff admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:19.224536 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1404/2794] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:12:19 2022] PUT /identity/v3/projects/17fc908900ca4ca5bc551b4b89708a0e/users/4a656a11adc346d8844efcd564599f3b/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:19.235426 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafed65-1e8e-4e0e-ad5c-963061bfc45f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:19.235718 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafed65-1e8e-4e0e-ad5c-963061bfc45f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:19.235995 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafed65-1e8e-4e0e-ad5c-963061bfc45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:19.266062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4cafed65-1e8e-4e0e-ad5c-963061bfc45f None None] MFA Rules not processed for user `4a656a11adc346d8844efcd564599f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:19.267328 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4cafed65-1e8e-4e0e-ad5c-963061bfc45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:19.348091 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1391/2795] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:19 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:19.360485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23d7ef-b104-418b-a73c-f86dd06bc4ff admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:19.362253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1b23d7ef-b104-418b-a73c-f86dd06bc4ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:19.452196 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23d7ef-b104-418b-a73c-f86dd06bc4ff admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:19.455987 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4346550-54a6-4cb7-8eaf-9a0dd326d742 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:19.456643 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4346550-54a6-4cb7-8eaf-9a0dd326d742 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:19.456931 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4346550-54a6-4cb7-8eaf-9a0dd326d742 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:19.457208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4346550-54a6-4cb7-8eaf-9a0dd326d742 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:19.467077 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4346550-54a6-4cb7-8eaf-9a0dd326d742 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:19.468518 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4346550-54a6-4cb7-8eaf-9a0dd326d742 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:19.479507 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1405/2796] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:12:19 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:12:19.903489 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafed65-1e8e-4e0e-ad5c-963061bfc45f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:19.968576 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafed65-1e8e-4e0e-ad5c-963061bfc45f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:19.973158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb964b-4014-494e-a8c5-d4acbb39d736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:19.973942 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb964b-4014-494e-a8c5-d4acbb39d736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:19.974222 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb964b-4014-494e-a8c5-d4acbb39d736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:19.974494 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb964b-4014-494e-a8c5-d4acbb39d736 admin admin] PATH_INFO: `/v3/users/4c9097fd72574b07bd22e6ab26b56f32` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:19.975715 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb964b-4014-494e-a8c5-d4acbb39d736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c9097fd72574b07bd22e6ab26b56f32)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:19.978116 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb964b-4014-494e-a8c5-d4acbb39d736 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:20.025639 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ddcb964b-4014-494e-a8c5-d4acbb39d736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c9097fd72574b07bd22e6ab26b56f32'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:20.026307 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ddcb964b-4014-494e-a8c5-d4acbb39d736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c9097fd72574b07bd22e6ab26b56f32'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:20.033620 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ddcb964b-4014-494e-a8c5-d4acbb39d736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c9097fd72574b07bd22e6ab26b56f32'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:20.040780 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:20.046133 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ddcb964b-4014-494e-a8c5-d4acbb39d736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c9097fd72574b07bd22e6ab26b56f32'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:20.046880 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ddcb964b-4014-494e-a8c5-d4acbb39d736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c9097fd72574b07bd22e6ab26b56f32'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:20.063657 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1392/2797] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:12:19 2022] DELETE /identity/v3/users/4c9097fd72574b07bd22e6ab26b56f32 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:20.068314 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18348e8b-1a48-48a1-85e5-6d95fd95d4a4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:20.068835 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18348e8b-1a48-48a1-85e5-6d95fd95d4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:20.069045 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18348e8b-1a48-48a1-85e5-6d95fd95d4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:20.086538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-18348e8b-1a48-48a1-85e5-6d95fd95d4a4 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:20.087822 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-18348e8b-1a48-48a1-85e5-6d95fd95d4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:20.131677 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:20.153398 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1406/2798] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:20.163635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb964b-4014-494e-a8c5-d4acbb39d736 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:20.166009 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ddcb964b-4014-494e-a8c5-d4acbb39d736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:20.249458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb964b-4014-494e-a8c5-d4acbb39d736 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:20.253079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b2ea6-dd49-45a0-bc47-bd5cbc322ab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:20.253715 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b2ea6-dd49-45a0-bc47-bd5cbc322ab0 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:20.253997 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b2ea6-dd49-45a0-bc47-bd5cbc322ab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:20.254274 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b2ea6-dd49-45a0-bc47-bd5cbc322ab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:20.264370 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b2ea6-dd49-45a0-bc47-bd5cbc322ab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:20.265803 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b2ea6-dd49-45a0-bc47-bd5cbc322ab0 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:20.276044 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1393/2799] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:12:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:12:20.568434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18348e8b-1a48-48a1-85e5-6d95fd95d4a4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:20.571711 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-18348e8b-1a48-48a1-85e5-6d95fd95d4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:20.590079 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18348e8b-1a48-48a1-85e5-6d95fd95d4a4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:20.595669 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:20.596981 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:20.597178 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:20.597470 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3 admin admin] PATH_INFO: `/v3/projects/f2295df09b8c4f8c9e5bcf569d783b1b` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:20.598994 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2295df09b8c4f8c9e5bcf569d783b1b)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:20.600857 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:20.649659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2295df09b8c4f8c9e5bcf569d783b1b', 'request_id': 'req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:20.657569 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2295df09b8c4f8c9e5bcf569d783b1b', 'request_id': 'req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:20.663904 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2295df09b8c4f8c9e5bcf569d783b1b', 'request_id': 'req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:20.674449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3 admin admin] The token cache is being invalidate because project f2295df09b8c4f8c9e5bcf569d783b1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:12:20.674765 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:20.675395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:20.676924 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1407/2800] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:12:20 2022] DELETE /identity/v3/projects/f2295df09b8c4f8c9e5bcf569d783b1b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:20.681669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b2ea6-dd49-45a0-bc47-bd5cbc322ab0 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:20.691294 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b2ea6-dd49-45a0-bc47-bd5cbc322ab0 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:20.747213 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f77c4c-ded1-40d0-9499-a92ffdd06dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:20.748049 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f77c4c-ded1-40d0-9499-a92ffdd06dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:20.748333 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f77c4c-ded1-40d0-9499-a92ffdd06dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:20.748616 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f77c4c-ded1-40d0-9499-a92ffdd06dfd admin admin] PATH_INFO: `/v3/users/4a656a11adc346d8844efcd564599f3b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:20.749801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f77c4c-ded1-40d0-9499-a92ffdd06dfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a656a11adc346d8844efcd564599f3b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:20.751749 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f77c4c-ded1-40d0-9499-a92ffdd06dfd admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:20.796851 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-15f77c4c-ded1-40d0-9499-a92ffdd06dfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a656a11adc346d8844efcd564599f3b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:20.797427 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-15f77c4c-ded1-40d0-9499-a92ffdd06dfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a656a11adc346d8844efcd564599f3b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:20.804445 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-15f77c4c-ded1-40d0-9499-a92ffdd06dfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a656a11adc346d8844efcd564599f3b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:20.813280 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:20.818178 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-15f77c4c-ded1-40d0-9499-a92ffdd06dfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a656a11adc346d8844efcd564599f3b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:20.819023 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-15f77c4c-ded1-40d0-9499-a92ffdd06dfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a656a11adc346d8844efcd564599f3b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:20.832179 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1394/2801] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:12:20 2022] DELETE /identity/v3/users/4a656a11adc346d8844efcd564599f3b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:21.111860 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:21.113990 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:21.130880 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ac06a-b7db-4ce1-94ed-5299f846e4f3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:21.191068 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbfa89-d00c-4f3b-8912-debe6d982974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:21.191948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbfa89-d00c-4f3b-8912-debe6d982974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:21.192233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbfa89-d00c-4f3b-8912-debe6d982974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:21.192512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbfa89-d00c-4f3b-8912-debe6d982974 admin admin] PATH_INFO: `/v3/projects/17fc908900ca4ca5bc551b4b89708a0e` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:21.193722 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbfa89-d00c-4f3b-8912-debe6d982974 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17fc908900ca4ca5bc551b4b89708a0e)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:21.195846 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbfa89-d00c-4f3b-8912-debe6d982974 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:21.242362 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-61fbfa89-d00c-4f3b-8912-debe6d982974 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17fc908900ca4ca5bc551b4b89708a0e', 'request_id': 'req-61fbfa89-d00c-4f3b-8912-debe6d982974'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:21.250284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-61fbfa89-d00c-4f3b-8912-debe6d982974 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17fc908900ca4ca5bc551b4b89708a0e', 'request_id': 'req-61fbfa89-d00c-4f3b-8912-debe6d982974'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:21.256566 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-61fbfa89-d00c-4f3b-8912-debe6d982974 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17fc908900ca4ca5bc551b4b89708a0e', 'request_id': 'req-61fbfa89-d00c-4f3b-8912-debe6d982974'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:21.267041 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-61fbfa89-d00c-4f3b-8912-debe6d982974 admin admin] The token cache is being invalidate because project 17fc908900ca4ca5bc551b4b89708a0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:12:21.267337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-61fbfa89-d00c-4f3b-8912-debe6d982974 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:21.268035 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-61fbfa89-d00c-4f3b-8912-debe6d982974 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:21.269639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1408/2802] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:12:21 2022] DELETE /identity/v3/projects/17fc908900ca4ca5bc551b4b89708a0e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:21.275333 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb33a5e-1514-4270-b935-993c376d3260 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:21.275859 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb33a5e-1514-4270-b935-993c376d3260 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:21.275986 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb33a5e-1514-4270-b935-993c376d3260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:21.295235 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-0eb33a5e-1514-4270-b935-993c376d3260 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:21.296429 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0eb33a5e-1514-4270-b935-993c376d3260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:21.358207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1395/2803] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:21.366155 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aba855-ff13-4300-ae3d-dbb22529a9d7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:21.366442 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aba855-ff13-4300-ae3d-dbb22529a9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:21.366874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aba855-ff13-4300-ae3d-dbb22529a9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:21.375196 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:21.389817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f5aba855-ff13-4300-ae3d-dbb22529a9d7 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:21.391065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f5aba855-ff13-4300-ae3d-dbb22529a9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:21.399830 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1409/2804] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:21.404195 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb33a5e-1514-4270-b935-993c376d3260 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:21.413881 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb33a5e-1514-4270-b935-993c376d3260 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:21.419358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb587d-355c-48c1-afb5-a4deace6aa7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:21.420005 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb587d-355c-48c1-afb5-a4deace6aa7d admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:21.420306 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb587d-355c-48c1-afb5-a4deace6aa7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:21.420584 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb587d-355c-48c1-afb5-a4deace6aa7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:21.421303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb587d-355c-48c1-afb5-a4deace6aa7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:12:21.421611 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb587d-355c-48c1-afb5-a4deace6aa7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:21.423254 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb587d-355c-48c1-afb5-a4deace6aa7d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:21.435028 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1396/2805] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:12:21.439708 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aba855-ff13-4300-ae3d-dbb22529a9d7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:21.450461 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aba855-ff13-4300-ae3d-dbb22529a9d7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:21.455134 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a072d75-bf01-4693-8634-cdedad8619d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:21.455783 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a072d75-bf01-4693-8634-cdedad8619d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:21.456061 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a072d75-bf01-4693-8634-cdedad8619d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:21.456328 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a072d75-bf01-4693-8634-cdedad8619d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:21.457005 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a072d75-bf01-4693-8634-cdedad8619d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:21.458934 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a072d75-bf01-4693-8634-cdedad8619d2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:21.485099 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1410/2806] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:12:21 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:12:21.489193 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb587d-355c-48c1-afb5-a4deace6aa7d admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:21.500173 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb587d-355c-48c1-afb5-a4deace6aa7d admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:21.559218 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7bbf0-daab-4516-b439-099a21b3a829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:21.559891 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7bbf0-daab-4516-b439-099a21b3a829 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:21.560176 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7bbf0-daab-4516-b439-099a21b3a829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:21.560453 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7bbf0-daab-4516-b439-099a21b3a829 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:21.561155 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7bbf0-daab-4516-b439-099a21b3a829 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:21.563099 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7bbf0-daab-4516-b439-099a21b3a829 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:21.582719 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1397/2807] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:12:21 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:12:21.587519 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a072d75-bf01-4693-8634-cdedad8619d2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:21.597731 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a072d75-bf01-4693-8634-cdedad8619d2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:21.603801 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797afcff-c54f-4054-a093-d6030eda7c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:21.604491 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797afcff-c54f-4054-a093-d6030eda7c23 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:21.604777 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797afcff-c54f-4054-a093-d6030eda7c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:21.605053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797afcff-c54f-4054-a093-d6030eda7c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:21.605784 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797afcff-c54f-4054-a093-d6030eda7c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:21.607413 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797afcff-c54f-4054-a093-d6030eda7c23 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:21.618496 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1411/2808] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:12:21.622655 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7bbf0-daab-4516-b439-099a21b3a829 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:21.633030 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7bbf0-daab-4516-b439-099a21b3a829 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:21.640825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bd38b-2c98-426c-bd2f-8d662c035d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:21.641546 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bd38b-2c98-426c-bd2f-8d662c035d78 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:21.641812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bd38b-2c98-426c-bd2f-8d662c035d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:21.642212 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bd38b-2c98-426c-bd2f-8d662c035d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:21.643160 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4bd38b-2c98-426c-bd2f-8d662c035d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:21.644769 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4bd38b-2c98-426c-bd2f-8d662c035d78 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:21.657788 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1398/2809] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:12:21.661939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bd38b-2c98-426c-bd2f-8d662c035d78 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:21.662107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797afcff-c54f-4054-a093-d6030eda7c23 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:21.663898 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-797afcff-c54f-4054-a093-d6030eda7c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:21.672897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bd38b-2c98-426c-bd2f-8d662c035d78 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:21.677679 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dfd37-2b8a-42cb-97e6-35db2416b2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:21.678755 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dfd37-2b8a-42cb-97e6-35db2416b2ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:21.679062 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dfd37-2b8a-42cb-97e6-35db2416b2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:21.679343 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dfd37-2b8a-42cb-97e6-35db2416b2ef admin admin] PATH_INFO: `/v3/projects/e4435f8226a74cf99d5a972390ba10b3/users/9e46c591ef6947cc92d8075975196bb1/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:21.695417 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6dfd37-2b8a-42cb-97e6-35db2416b2ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4435f8226a74cf99d5a972390ba10b3, user_id=9e46c591ef6947cc92d8075975196bb1, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:21.698223 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6dfd37-2b8a-42cb-97e6-35db2416b2ef admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:21.712669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1399/2810] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:12:21 2022] PUT /identity/v3/projects/e4435f8226a74cf99d5a972390ba10b3/users/9e46c591ef6947cc92d8075975196bb1/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:12:21.723873 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e6c90-9645-47a5-8946-109ec0120371 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:21.724250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e6c90-9645-47a5-8946-109ec0120371 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:21.724703 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e6c90-9645-47a5-8946-109ec0120371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:21.736462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797afcff-c54f-4054-a093-d6030eda7c23 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:21.752826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-af2e6c90-9645-47a5-8946-109ec0120371 None None] MFA Rules not processed for user `9e46c591ef6947cc92d8075975196bb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:21.754020 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-af2e6c90-9645-47a5-8946-109ec0120371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:21.767182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:21.802330 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686d183-12f6-4f6c-a0fe-e21024943884 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '880d0299c38c4eb897c669a512e599be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:21.803172 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686d183-12f6-4f6c-a0fe-e21024943884 service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:21.803455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686d183-12f6-4f6c-a0fe-e21024943884 service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:21.803733 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686d183-12f6-4f6c-a0fe-e21024943884 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:21.805799 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b686d183-12f6-4f6c-a0fe-e21024943884 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:21.817626 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686d183-12f6-4f6c-a0fe-e21024943884 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:21.819407 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686d183-12f6-4f6c-a0fe-e21024943884 service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:21.821459 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1400/2811] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:12:21.825953 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e6c90-9645-47a5-8946-109ec0120371 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:21.830440 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1412/2812] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:12:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:12:21.893021 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e6c90-9645-47a5-8946-109ec0120371 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:21.897897 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b83c18-b15c-46a7-9fcd-839f00a183a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:21.899286 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b83c18-b15c-46a7-9fcd-839f00a183a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:21.899754 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b83c18-b15c-46a7-9fcd-839f00a183a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:21.900045 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b83c18-b15c-46a7-9fcd-839f00a183a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:21.900736 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b83c18-b15c-46a7-9fcd-839f00a183a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:21.902658 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b83c18-b15c-46a7-9fcd-839f00a183a5 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:21.928641 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1401/2813] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:12:21 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:12:21.932818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686d183-12f6-4f6c-a0fe-e21024943884 service placement] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:21.943065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686d183-12f6-4f6c-a0fe-e21024943884 service placement] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:22.001045 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:22.008181 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bab9ed-4be4-4507-8339-fd46b84280bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:22.009614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bab9ed-4be4-4507-8339-fd46b84280bd admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:22.009614 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bab9ed-4be4-4507-8339-fd46b84280bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:22.010346 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bab9ed-4be4-4507-8339-fd46b84280bd admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:22.010970 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bab9ed-4be4-4507-8339-fd46b84280bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:22.013470 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bab9ed-4be4-4507-8339-fd46b84280bd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:22.038733 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1413/2814] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:12:21 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:12:22.039945 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b83c18-b15c-46a7-9fcd-839f00a183a5 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:22.050699 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b83c18-b15c-46a7-9fcd-839f00a183a5 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:22.055279 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de4c9e-398d-46a3-923f-ca2315319a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:22.055958 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de4c9e-398d-46a3-923f-ca2315319a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:22.056245 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de4c9e-398d-46a3-923f-ca2315319a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:22.056525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de4c9e-398d-46a3-923f-ca2315319a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:22.057244 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de4c9e-398d-46a3-923f-ca2315319a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:22.058862 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de4c9e-398d-46a3-923f-ca2315319a12 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:22.069532 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1402/2815] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:12:22.074601 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bab9ed-4be4-4507-8339-fd46b84280bd admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:22.084545 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bab9ed-4be4-4507-8339-fd46b84280bd admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:22.090076 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271b2f3-0ab2-4600-a9bb-39818d52bb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:22.090974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271b2f3-0ab2-4600-a9bb-39818d52bb90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:22.091252 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271b2f3-0ab2-4600-a9bb-39818d52bb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:22.091524 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271b2f3-0ab2-4600-a9bb-39818d52bb90 admin admin] PATH_INFO: `/v3/projects/8941a64d92f542afa88e9377afe19227/users/27a1e935242d4c3a8cec4c3ee8c7aa07/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:22.106177 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271b2f3-0ab2-4600-a9bb-39818d52bb90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8941a64d92f542afa88e9377afe19227, user_id=27a1e935242d4c3a8cec4c3ee8c7aa07, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:22.108861 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271b2f3-0ab2-4600-a9bb-39818d52bb90 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:22.122924 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1414/2816] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:12:22 2022] PUT /identity/v3/projects/8941a64d92f542afa88e9377afe19227/users/27a1e935242d4c3a8cec4c3ee8c7aa07/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:22.126833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de4c9e-398d-46a3-923f-ca2315319a12 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:22.138076 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de4c9e-398d-46a3-923f-ca2315319a12 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:22.194213 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2752edb-1592-4b73-9591-0e4b2bba8c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:22.194914 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2752edb-1592-4b73-9591-0e4b2bba8c40 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:22.195284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2752edb-1592-4b73-9591-0e4b2bba8c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:22.195555 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2752edb-1592-4b73-9591-0e4b2bba8c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:22.196267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2752edb-1592-4b73-9591-0e4b2bba8c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:22.198626 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2752edb-1592-4b73-9591-0e4b2bba8c40 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:22.208669 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1403/2817] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:12:22.213384 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271b2f3-0ab2-4600-a9bb-39818d52bb90 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:22.223642 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271b2f3-0ab2-4600-a9bb-39818d52bb90 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:22.228300 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7adc6-8541-4c20-bb89-d1dc23ed1418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:22.229180 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7adc6-8541-4c20-bb89-d1dc23ed1418 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:22.229467 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7adc6-8541-4c20-bb89-d1dc23ed1418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:22.229745 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7adc6-8541-4c20-bb89-d1dc23ed1418 admin admin] PATH_INFO: `/v3/projects/8941a64d92f542afa88e9377afe19227/users/27a1e935242d4c3a8cec4c3ee8c7aa07/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:22.231764 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7adc6-8541-4c20-bb89-d1dc23ed1418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8941a64d92f542afa88e9377afe19227, user_id=27a1e935242d4c3a8cec4c3ee8c7aa07, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:22.234410 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7adc6-8541-4c20-bb89-d1dc23ed1418 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:22.245558 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1415/2818] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:12:22 2022] PUT /identity/v3/projects/8941a64d92f542afa88e9377afe19227/users/27a1e935242d4c3a8cec4c3ee8c7aa07/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:22.255537 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509b83b-586d-4cfc-a018-d705592b5148 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:22.255833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509b83b-586d-4cfc-a018-d705592b5148 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:22.256116 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509b83b-586d-4cfc-a018-d705592b5148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:22.284339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b509b83b-586d-4cfc-a018-d705592b5148 None None] MFA Rules not processed for user `27a1e935242d4c3a8cec4c3ee8c7aa07`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:22.286839 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b509b83b-586d-4cfc-a018-d705592b5148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:22.301069 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:22.359470 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1404/2819] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:22 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:22.372875 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7adc6-8541-4c20-bb89-d1dc23ed1418 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:22.374605 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f3e7adc6-8541-4c20-bb89-d1dc23ed1418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:22.434982 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7adc6-8541-4c20-bb89-d1dc23ed1418 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:22.438872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeb0da-4aa7-4436-86a0-42ba65fa23e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:22.439524 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeb0da-4aa7-4436-86a0-42ba65fa23e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:22.439811 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeb0da-4aa7-4436-86a0-42ba65fa23e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:22.440128 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeb0da-4aa7-4436-86a0-42ba65fa23e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:22.449806 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aeb0da-4aa7-4436-86a0-42ba65fa23e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:22.451201 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aeb0da-4aa7-4436-86a0-42ba65fa23e6 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:22.460449 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1416/2820] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:12:22 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:12:22.532261 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509b83b-586d-4cfc-a018-d705592b5148 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:22.542496 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509b83b-586d-4cfc-a018-d705592b5148 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:22.546283 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361a9f4-1ffb-423d-a04e-74ed663c971a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:22.546965 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361a9f4-1ffb-423d-a04e-74ed663c971a service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:22.547247 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361a9f4-1ffb-423d-a04e-74ed663c971a service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:22.547525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361a9f4-1ffb-423d-a04e-74ed663c971a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:22.604394 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361a9f4-1ffb-423d-a04e-74ed663c971a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:22.605908 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361a9f4-1ffb-423d-a04e-74ed663c971a service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:22.616587 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1405/2821] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:12:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:12:25.336779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeb0da-4aa7-4436-86a0-42ba65fa23e6 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:25.406986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeb0da-4aa7-4436-86a0-42ba65fa23e6 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:25.413885 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3939f65-c374-409b-92d3-949491cd7045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:25.414817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3939f65-c374-409b-92d3-949491cd7045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:25.415655 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3939f65-c374-409b-92d3-949491cd7045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:25.416647 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3939f65-c374-409b-92d3-949491cd7045 admin admin] PATH_INFO: `/v3/users/9e46c591ef6947cc92d8075975196bb1` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:25.417952 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3939f65-c374-409b-92d3-949491cd7045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e46c591ef6947cc92d8075975196bb1)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:25.419969 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3939f65-c374-409b-92d3-949491cd7045 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:25.474727 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d3939f65-c374-409b-92d3-949491cd7045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e46c591ef6947cc92d8075975196bb1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:25.475335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d3939f65-c374-409b-92d3-949491cd7045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e46c591ef6947cc92d8075975196bb1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:25.475940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d3939f65-c374-409b-92d3-949491cd7045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e46c591ef6947cc92d8075975196bb1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:25.484359 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d3939f65-c374-409b-92d3-949491cd7045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e46c591ef6947cc92d8075975196bb1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:25.486587 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:25.496278 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d3939f65-c374-409b-92d3-949491cd7045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e46c591ef6947cc92d8075975196bb1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:25.519241 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1417/2822] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:12:25 2022] DELETE /identity/v3/users/9e46c591ef6947cc92d8075975196bb1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:25.523801 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac28660-8c46-4791-906f-0d955113f795 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:25.524297 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac28660-8c46-4791-906f-0d955113f795 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:25.524491 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac28660-8c46-4791-906f-0d955113f795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:25.543656 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1ac28660-8c46-4791-906f-0d955113f795 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:25.544862 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1ac28660-8c46-4791-906f-0d955113f795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:25.607380 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1406/2823] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:25.617182 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3939f65-c374-409b-92d3-949491cd7045 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:25.618698 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d3939f65-c374-409b-92d3-949491cd7045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:25.676057 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3939f65-c374-409b-92d3-949491cd7045 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:25.679781 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3905c-2004-4697-b550-ce66780f6a67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:25.680455 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3905c-2004-4697-b550-ce66780f6a67 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:25.681011 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3905c-2004-4697-b550-ce66780f6a67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:25.681519 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3905c-2004-4697-b550-ce66780f6a67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:25.690777 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3905c-2004-4697-b550-ce66780f6a67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:25.692299 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3905c-2004-4697-b550-ce66780f6a67 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:25.702235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1418/2824] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:12:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:12:25.974825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac28660-8c46-4791-906f-0d955113f795 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:25.976402 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1ac28660-8c46-4791-906f-0d955113f795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:25.990462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac28660-8c46-4791-906f-0d955113f795 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:25.996026 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c327a-1925-43d0-84c1-35701edf4b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:25.996889 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c327a-1925-43d0-84c1-35701edf4b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:25.997174 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c327a-1925-43d0-84c1-35701edf4b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:25.997449 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c327a-1925-43d0-84c1-35701edf4b84 admin admin] PATH_INFO: `/v3/projects/e4435f8226a74cf99d5a972390ba10b3` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:25.998746 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c327a-1925-43d0-84c1-35701edf4b84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4435f8226a74cf99d5a972390ba10b3)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:26.000635 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c327a-1925-43d0-84c1-35701edf4b84 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:26.047783 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:26.050944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6c5c327a-1925-43d0-84c1-35701edf4b84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4435f8226a74cf99d5a972390ba10b3', 'request_id': 'req-6c5c327a-1925-43d0-84c1-35701edf4b84'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:26.059193 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6c5c327a-1925-43d0-84c1-35701edf4b84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4435f8226a74cf99d5a972390ba10b3', 'request_id': 'req-6c5c327a-1925-43d0-84c1-35701edf4b84'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:26.065841 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6c5c327a-1925-43d0-84c1-35701edf4b84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4435f8226a74cf99d5a972390ba10b3', 'request_id': 'req-6c5c327a-1925-43d0-84c1-35701edf4b84'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:26.077076 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6c5c327a-1925-43d0-84c1-35701edf4b84 admin admin] The token cache is being invalidate because project e4435f8226a74cf99d5a972390ba10b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:12:26.077366 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6c5c327a-1925-43d0-84c1-35701edf4b84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:26.078006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6c5c327a-1925-43d0-84c1-35701edf4b84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:26.079590 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1407/2825] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:12:25 2022] DELETE /identity/v3/projects/e4435f8226a74cf99d5a972390ba10b3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:26.083732 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3905c-2004-4697-b550-ce66780f6a67 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:26.093590 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3905c-2004-4697-b550-ce66780f6a67 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:26.151479 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b3900-de33-4926-9c09-4b0055187214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:26.152356 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b3900-de33-4926-9c09-4b0055187214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:26.152644 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b3900-de33-4926-9c09-4b0055187214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:26.152925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b3900-de33-4926-9c09-4b0055187214 admin admin] PATH_INFO: `/v3/users/27a1e935242d4c3a8cec4c3ee8c7aa07` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:26.154369 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b3900-de33-4926-9c09-4b0055187214 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27a1e935242d4c3a8cec4c3ee8c7aa07)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:26.156310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b3900-de33-4926-9c09-4b0055187214 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:26.201726 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-035b3900-de33-4926-9c09-4b0055187214 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27a1e935242d4c3a8cec4c3ee8c7aa07'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:26.202898 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-035b3900-de33-4926-9c09-4b0055187214 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27a1e935242d4c3a8cec4c3ee8c7aa07'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:26.203499 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-035b3900-de33-4926-9c09-4b0055187214 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27a1e935242d4c3a8cec4c3ee8c7aa07'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:26.210909 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-035b3900-de33-4926-9c09-4b0055187214 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27a1e935242d4c3a8cec4c3ee8c7aa07'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:26.219107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-035b3900-de33-4926-9c09-4b0055187214 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27a1e935242d4c3a8cec4c3ee8c7aa07'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:26.230081 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:26.239490 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1419/2826] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:12:26 2022] DELETE /identity/v3/users/27a1e935242d4c3a8cec4c3ee8c7aa07 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:26.531051 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c327a-1925-43d0-84c1-35701edf4b84 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:26.532540 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6c5c327a-1925-43d0-84c1-35701edf4b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:26.555850 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c327a-1925-43d0-84c1-35701edf4b84 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:26.615579 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee036477-6aef-4598-9373-316db03b4626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:26.616387 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee036477-6aef-4598-9373-316db03b4626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:26.616665 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee036477-6aef-4598-9373-316db03b4626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:26.616941 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee036477-6aef-4598-9373-316db03b4626 admin admin] PATH_INFO: `/v3/projects/8941a64d92f542afa88e9377afe19227` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:26.618783 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee036477-6aef-4598-9373-316db03b4626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8941a64d92f542afa88e9377afe19227)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:26.620758 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee036477-6aef-4598-9373-316db03b4626 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:26.666691 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ee036477-6aef-4598-9373-316db03b4626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8941a64d92f542afa88e9377afe19227', 'request_id': 'req-ee036477-6aef-4598-9373-316db03b4626'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:26.673716 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ee036477-6aef-4598-9373-316db03b4626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8941a64d92f542afa88e9377afe19227', 'request_id': 'req-ee036477-6aef-4598-9373-316db03b4626'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:26.679936 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ee036477-6aef-4598-9373-316db03b4626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8941a64d92f542afa88e9377afe19227', 'request_id': 'req-ee036477-6aef-4598-9373-316db03b4626'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:26.691078 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ee036477-6aef-4598-9373-316db03b4626 admin admin] The token cache is being invalidate because project 8941a64d92f542afa88e9377afe19227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:12:26.691374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ee036477-6aef-4598-9373-316db03b4626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:26.692005 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ee036477-6aef-4598-9373-316db03b4626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:26.693525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1408/2827] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:12:26 2022] DELETE /identity/v3/projects/8941a64d92f542afa88e9377afe19227 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:26.699293 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453515d-53cc-4c22-b5ee-377a80a54263 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:26.699760 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453515d-53cc-4c22-b5ee-377a80a54263 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:26.699859 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453515d-53cc-4c22-b5ee-377a80a54263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:26.719606 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9453515d-53cc-4c22-b5ee-377a80a54263 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:26.720796 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9453515d-53cc-4c22-b5ee-377a80a54263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:26.786235 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1420/2828] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:12:26.795546 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677246c0-4722-4aad-b47c-cdd44d1ddc85 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:26.795833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677246c0-4722-4aad-b47c-cdd44d1ddc85 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:26.796111 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677246c0-4722-4aad-b47c-cdd44d1ddc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:26.816386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-677246c0-4722-4aad-b47c-cdd44d1ddc85 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:26.817615 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-677246c0-4722-4aad-b47c-cdd44d1ddc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:26.826532 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1409/2829] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:26.830748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453515d-53cc-4c22-b5ee-377a80a54263 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:26.840843 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453515d-53cc-4c22-b5ee-377a80a54263 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:26.845329 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771dffe-3928-483a-9cdf-2294e876822b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:26.845986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771dffe-3928-483a-9cdf-2294e876822b admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:26.846271 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771dffe-3928-483a-9cdf-2294e876822b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:26.846548 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771dffe-3928-483a-9cdf-2294e876822b admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:26.847305 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771dffe-3928-483a-9cdf-2294e876822b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:12:26.847622 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771dffe-3928-483a-9cdf-2294e876822b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:26.849769 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771dffe-3928-483a-9cdf-2294e876822b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:26.861924 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1421/2830] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:12:26.866234 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677246c0-4722-4aad-b47c-cdd44d1ddc85 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:26.876012 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677246c0-4722-4aad-b47c-cdd44d1ddc85 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:26.880587 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f4f8b-ede8-4573-959c-20d568598b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:26.881222 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f4f8b-ede8-4573-959c-20d568598b2f admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:26.881503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f4f8b-ede8-4573-959c-20d568598b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:26.881773 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f4f8b-ede8-4573-959c-20d568598b2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:26.882450 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f4f8b-ede8-4573-959c-20d568598b2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:26.885122 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f4f8b-ede8-4573-959c-20d568598b2f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:26.910161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1410/2831] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:12:26 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:12:26.914252 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771dffe-3928-483a-9cdf-2294e876822b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:26.924531 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771dffe-3928-483a-9cdf-2294e876822b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:26.986335 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5a54b-3fe7-4d66-b647-1401260ed7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:26.987053 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5a54b-3fe7-4d66-b647-1401260ed7c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:26.987336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5a54b-3fe7-4d66-b647-1401260ed7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:26.987612 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5a54b-3fe7-4d66-b647-1401260ed7c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:26.988341 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5a54b-3fe7-4d66-b647-1401260ed7c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:26.990265 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5a54b-3fe7-4d66-b647-1401260ed7c5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:27.012289 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1422/2832] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:12:26 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:12:27.016381 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f4f8b-ede8-4573-959c-20d568598b2f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:27.028071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f4f8b-ede8-4573-959c-20d568598b2f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:27.033130 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330eef3-3f3a-4e3a-8e27-1a708e5a718f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:27.033814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330eef3-3f3a-4e3a-8e27-1a708e5a718f admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:27.034097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330eef3-3f3a-4e3a-8e27-1a708e5a718f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:27.034374 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330eef3-3f3a-4e3a-8e27-1a708e5a718f admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:27.035135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330eef3-3f3a-4e3a-8e27-1a708e5a718f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:27.036822 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330eef3-3f3a-4e3a-8e27-1a708e5a718f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:27.047322 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1411/2833] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:12:27.051140 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5a54b-3fe7-4d66-b647-1401260ed7c5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:27.061265 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5a54b-3fe7-4d66-b647-1401260ed7c5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:27.065878 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b17ec0-d13e-4a53-b9e0-e5be3e1daa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:27.066533 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b17ec0-d13e-4a53-b9e0-e5be3e1daa16 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:27.066850 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b17ec0-d13e-4a53-b9e0-e5be3e1daa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:27.067131 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b17ec0-d13e-4a53-b9e0-e5be3e1daa16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:27.067858 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b17ec0-d13e-4a53-b9e0-e5be3e1daa16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:27.069434 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b17ec0-d13e-4a53-b9e0-e5be3e1daa16 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:27.080411 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1423/2834] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:12:27.084378 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330eef3-3f3a-4e3a-8e27-1a708e5a718f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:27.095654 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330eef3-3f3a-4e3a-8e27-1a708e5a718f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:27.100314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4b31f-f092-400f-963e-2847e5b966d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:27.101182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4b31f-f092-400f-963e-2847e5b966d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:27.101464 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4b31f-f092-400f-963e-2847e5b966d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:27.101741 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4b31f-f092-400f-963e-2847e5b966d9 admin admin] PATH_INFO: `/v3/projects/fcfea27d5f0e4858871c60e9379b4608/users/72e1e3b1ff944dc0b5d547b3f307179a/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:27.121027 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4b31f-f092-400f-963e-2847e5b966d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcfea27d5f0e4858871c60e9379b4608, user_id=72e1e3b1ff944dc0b5d547b3f307179a, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:27.127364 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4b31f-f092-400f-963e-2847e5b966d9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:27.134861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:27.145790 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1412/2835] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:12:27 2022] PUT /identity/v3/projects/fcfea27d5f0e4858871c60e9379b4608/users/72e1e3b1ff944dc0b5d547b3f307179a/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:27.151318 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac040c2f-256a-44d1-ba3f-be61809babcc None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:27.151613 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac040c2f-256a-44d1-ba3f-be61809babcc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:27.151898 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac040c2f-256a-44d1-ba3f-be61809babcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:27.171414 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ac040c2f-256a-44d1-ba3f-be61809babcc None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:27.172621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ac040c2f-256a-44d1-ba3f-be61809babcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:27.227480 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:27.237345 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1424/2836] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:27.247140 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4b31f-f092-400f-963e-2847e5b966d9 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:27.248906 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1fb4b31f-f092-400f-963e-2847e5b966d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:27.304904 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4b31f-f092-400f-963e-2847e5b966d9 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:27.308614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86060f6-0d13-4c4c-817a-87845270d52b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:27.309281 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86060f6-0d13-4c4c-817a-87845270d52b service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:27.309558 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86060f6-0d13-4c4c-817a-87845270d52b service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:27.309832 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86060f6-0d13-4c4c-817a-87845270d52b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:27.319386 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86060f6-0d13-4c4c-817a-87845270d52b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:27.320826 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86060f6-0d13-4c4c-817a-87845270d52b service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:27.331204 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1413/2837] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:12:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:12:27.960304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3609a8-4374-4a6d-8b9a-930fe6abcb96 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:27.960734 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3609a8-4374-4a6d-8b9a-930fe6abcb96 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:27.961028 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3609a8-4374-4a6d-8b9a-930fe6abcb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:27.987800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0a3609a8-4374-4a6d-8b9a-930fe6abcb96 None None] MFA Rules not processed for user `72e1e3b1ff944dc0b5d547b3f307179a`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:27.988987 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0a3609a8-4374-4a6d-8b9a-930fe6abcb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:28.057474 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1425/2838] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:28.061983 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86060f6-0d13-4c4c-817a-87845270d52b service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:28.072136 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86060f6-0d13-4c4c-817a-87845270d52b service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:28.077303 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e8e4b-283f-4245-a96d-236b4f22354d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:28.077944 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e8e4b-283f-4245-a96d-236b4f22354d admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:28.078227 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e8e4b-283f-4245-a96d-236b4f22354d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:28.078506 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e8e4b-283f-4245-a96d-236b4f22354d admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:28.079236 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e8e4b-283f-4245-a96d-236b4f22354d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:28.081156 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e8e4b-283f-4245-a96d-236b4f22354d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:28.108421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1414/2839] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:12:28 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:12:28.112587 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3609a8-4374-4a6d-8b9a-930fe6abcb96 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:28.125728 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3609a8-4374-4a6d-8b9a-930fe6abcb96 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:28.184210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975c797-629c-4c6e-8a96-2df49d5d97ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:28.184889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975c797-629c-4c6e-8a96-2df49d5d97ac admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:28.185175 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975c797-629c-4c6e-8a96-2df49d5d97ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:28.185458 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975c797-629c-4c6e-8a96-2df49d5d97ac admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:28.186157 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975c797-629c-4c6e-8a96-2df49d5d97ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:28.188124 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975c797-629c-4c6e-8a96-2df49d5d97ac admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:28.208452 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1426/2840] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:12:28 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:12:28.213268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e8e4b-283f-4245-a96d-236b4f22354d admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:28.223561 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e8e4b-283f-4245-a96d-236b4f22354d admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:28.228454 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bda625-d739-455c-8458-afb155da6474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:28.229136 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bda625-d739-455c-8458-afb155da6474 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:28.229421 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bda625-d739-455c-8458-afb155da6474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:28.229703 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bda625-d739-455c-8458-afb155da6474 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:28.230424 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bda625-d739-455c-8458-afb155da6474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:28.232056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bda625-d739-455c-8458-afb155da6474 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:28.242597 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1415/2841] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:12:28.247201 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975c797-629c-4c6e-8a96-2df49d5d97ac admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:28.256970 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975c797-629c-4c6e-8a96-2df49d5d97ac admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:28.263178 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcf898-0d00-4aba-9697-8695e4d19231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:28.264058 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcf898-0d00-4aba-9697-8695e4d19231 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:28.264347 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcf898-0d00-4aba-9697-8695e4d19231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:28.264630 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcf898-0d00-4aba-9697-8695e4d19231 admin admin] PATH_INFO: `/v3/projects/f9a0c68e37d24845be24ba44937102ad/users/600f1bbf62b2434ba1c0265c40017e63/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:28.279616 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdcf898-0d00-4aba-9697-8695e4d19231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a0c68e37d24845be24ba44937102ad, user_id=600f1bbf62b2434ba1c0265c40017e63, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:28.282277 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdcf898-0d00-4aba-9697-8695e4d19231 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:28.295485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1427/2842] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:12:28 2022] PUT /identity/v3/projects/f9a0c68e37d24845be24ba44937102ad/users/600f1bbf62b2434ba1c0265c40017e63/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:28.299487 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bda625-d739-455c-8458-afb155da6474 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:28.310520 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bda625-d739-455c-8458-afb155da6474 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:28.367918 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff02281-3385-4931-984f-f64f153d0615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:28.368835 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff02281-3385-4931-984f-f64f153d0615 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:28.369137 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff02281-3385-4931-984f-f64f153d0615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:28.369415 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff02281-3385-4931-984f-f64f153d0615 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:28.370135 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff02281-3385-4931-984f-f64f153d0615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:28.371788 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff02281-3385-4931-984f-f64f153d0615 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:28.381696 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1416/2843] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:12:28.385784 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcf898-0d00-4aba-9697-8695e4d19231 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:28.395896 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcf898-0d00-4aba-9697-8695e4d19231 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:28.401246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15058f-29e4-48cf-a72e-3b9c76babad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:28.402113 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15058f-29e4-48cf-a72e-3b9c76babad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:28.402395 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15058f-29e4-48cf-a72e-3b9c76babad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:28.402706 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15058f-29e4-48cf-a72e-3b9c76babad9 admin admin] PATH_INFO: `/v3/projects/f9a0c68e37d24845be24ba44937102ad/users/600f1bbf62b2434ba1c0265c40017e63/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:28.404904 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15058f-29e4-48cf-a72e-3b9c76babad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a0c68e37d24845be24ba44937102ad, user_id=600f1bbf62b2434ba1c0265c40017e63, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:28.407570 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15058f-29e4-48cf-a72e-3b9c76babad9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:28.418690 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1428/2844] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:12:28 2022] PUT /identity/v3/projects/f9a0c68e37d24845be24ba44937102ad/users/600f1bbf62b2434ba1c0265c40017e63/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:29.092333 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e8424-c16c-489a-bf8d-3cc82b82a020 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:29.092874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e8424-c16c-489a-bf8d-3cc82b82a020 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:29.092874 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e8424-c16c-489a-bf8d-3cc82b82a020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:29.127359 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-204e8424-c16c-489a-bf8d-3cc82b82a020 None None] MFA Rules not processed for user `600f1bbf62b2434ba1c0265c40017e63`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:29.129112 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-204e8424-c16c-489a-bf8d-3cc82b82a020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:29.208284 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:29.222714 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1417/2845] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:29.238038 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15058f-29e4-48cf-a72e-3b9c76babad9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:29.240037 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-da15058f-29e4-48cf-a72e-3b9c76babad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:29.288953 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:29.301766 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15058f-29e4-48cf-a72e-3b9c76babad9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:29.305704 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda6293-3cc2-4205-94e1-e310f8295395 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:29.306970 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda6293-3cc2-4205-94e1-e310f8295395 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:29.307275 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda6293-3cc2-4205-94e1-e310f8295395 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:29.307512 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda6293-3cc2-4205-94e1-e310f8295395 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:29.366852 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda6293-3cc2-4205-94e1-e310f8295395 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:29.369121 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda6293-3cc2-4205-94e1-e310f8295395 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:29.378347 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1429/2846] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:12:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:12:30.734876 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e8424-c16c-489a-bf8d-3cc82b82a020 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:30.736278 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-204e8424-c16c-489a-bf8d-3cc82b82a020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:30.749097 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e8424-c16c-489a-bf8d-3cc82b82a020 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:30.752804 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7946e92-f9c6-4dc3-a320-27f8afd95456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:30.753449 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7946e92-f9c6-4dc3-a320-27f8afd95456 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:30.753760 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7946e92-f9c6-4dc3-a320-27f8afd95456 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:30.754040 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7946e92-f9c6-4dc3-a320-27f8afd95456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:30.763822 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7946e92-f9c6-4dc3-a320-27f8afd95456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:30.765276 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7946e92-f9c6-4dc3-a320-27f8afd95456 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:30.775804 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1418/2847] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:12:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:12:39.374280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=29 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:40.770781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=10 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:43.117651 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda6293-3cc2-4205-94e1-e310f8295395 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:43.119863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6dda6293-3cc2-4205-94e1-e310f8295395 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:43.203591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda6293-3cc2-4205-94e1-e310f8295395 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:43.208659 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0584ccc-ee8c-4c3c-a372-02a18eabc26d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '880d0299c38c4eb897c669a512e599be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:43.209310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0584ccc-ee8c-4c3c-a372-02a18eabc26d service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:43.209601 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0584ccc-ee8c-4c3c-a372-02a18eabc26d service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:43.209884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0584ccc-ee8c-4c3c-a372-02a18eabc26d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:43.211735 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e0584ccc-ee8c-4c3c-a372-02a18eabc26d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:43.223464 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0584ccc-ee8c-4c3c-a372-02a18eabc26d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:43.225101 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0584ccc-ee8c-4c3c-a372-02a18eabc26d service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:43.236820 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1430/2848] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:12:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:12:43.708181 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7946e92-f9c6-4dc3-a320-27f8afd95456 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:43.775420 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7946e92-f9c6-4dc3-a320-27f8afd95456 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:43.780337 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7b1db-3314-409d-8eb0-bfb8f0661a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:43.781202 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7b1db-3314-409d-8eb0-bfb8f0661a20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:43.781483 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7b1db-3314-409d-8eb0-bfb8f0661a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:43.781761 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7b1db-3314-409d-8eb0-bfb8f0661a20 admin admin] PATH_INFO: `/v3/users/72e1e3b1ff944dc0b5d547b3f307179a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:43.783001 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7b1db-3314-409d-8eb0-bfb8f0661a20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72e1e3b1ff944dc0b5d547b3f307179a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:43.784912 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7b1db-3314-409d-8eb0-bfb8f0661a20 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:43.832762 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7af7b1db-3314-409d-8eb0-bfb8f0661a20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72e1e3b1ff944dc0b5d547b3f307179a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:43.833418 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7af7b1db-3314-409d-8eb0-bfb8f0661a20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72e1e3b1ff944dc0b5d547b3f307179a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:43.840900 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7af7b1db-3314-409d-8eb0-bfb8f0661a20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72e1e3b1ff944dc0b5d547b3f307179a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:43.853524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7af7b1db-3314-409d-8eb0-bfb8f0661a20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72e1e3b1ff944dc0b5d547b3f307179a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:43.854339 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7af7b1db-3314-409d-8eb0-bfb8f0661a20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72e1e3b1ff944dc0b5d547b3f307179a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:43.866750 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1419/2849] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:12:43 2022] DELETE /identity/v3/users/72e1e3b1ff944dc0b5d547b3f307179a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:44.041027 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0584ccc-ee8c-4c3c-a372-02a18eabc26d service placement] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:44.051743 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0584ccc-ee8c-4c3c-a372-02a18eabc26d service placement] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:44.113367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5216f-db3a-4251-8a29-a98a9a2883d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:44.114227 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5216f-db3a-4251-8a29-a98a9a2883d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:44.114510 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5216f-db3a-4251-8a29-a98a9a2883d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:44.114818 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5216f-db3a-4251-8a29-a98a9a2883d9 admin admin] PATH_INFO: `/v3/projects/fcfea27d5f0e4858871c60e9379b4608` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:44.116023 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5216f-db3a-4251-8a29-a98a9a2883d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcfea27d5f0e4858871c60e9379b4608)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:44.117913 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5216f-db3a-4251-8a29-a98a9a2883d9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:44.166889 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a8e5216f-db3a-4251-8a29-a98a9a2883d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcfea27d5f0e4858871c60e9379b4608', 'request_id': 'req-a8e5216f-db3a-4251-8a29-a98a9a2883d9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:44.173920 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a8e5216f-db3a-4251-8a29-a98a9a2883d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcfea27d5f0e4858871c60e9379b4608', 'request_id': 'req-a8e5216f-db3a-4251-8a29-a98a9a2883d9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:44.180096 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a8e5216f-db3a-4251-8a29-a98a9a2883d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcfea27d5f0e4858871c60e9379b4608', 'request_id': 'req-a8e5216f-db3a-4251-8a29-a98a9a2883d9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:44.190663 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a8e5216f-db3a-4251-8a29-a98a9a2883d9 admin admin] The token cache is being invalidate because project fcfea27d5f0e4858871c60e9379b4608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:12:44.190957 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a8e5216f-db3a-4251-8a29-a98a9a2883d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:44.191603 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a8e5216f-db3a-4251-8a29-a98a9a2883d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:44.193432 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1431/2850] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:12:44 2022] DELETE /identity/v3/projects/fcfea27d5f0e4858871c60e9379b4608 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:44.197737 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7b1db-3314-409d-8eb0-bfb8f0661a20 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:44.199164 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7af7b1db-3314-409d-8eb0-bfb8f0661a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:44.212736 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7b1db-3314-409d-8eb0-bfb8f0661a20 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:44.270071 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89a9dc-2bd0-448e-9a6d-473e0426c4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:44.270982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89a9dc-2bd0-448e-9a6d-473e0426c4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:44.271269 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89a9dc-2bd0-448e-9a6d-473e0426c4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:44.271547 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89a9dc-2bd0-448e-9a6d-473e0426c4f2 admin admin] PATH_INFO: `/v3/users/600f1bbf62b2434ba1c0265c40017e63` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:44.272841 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89a9dc-2bd0-448e-9a6d-473e0426c4f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=600f1bbf62b2434ba1c0265c40017e63)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:44.275033 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89a9dc-2bd0-448e-9a6d-473e0426c4f2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:44.300337 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:44.329337 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-de89a9dc-2bd0-448e-9a6d-473e0426c4f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '600f1bbf62b2434ba1c0265c40017e63'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:44.330309 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-de89a9dc-2bd0-448e-9a6d-473e0426c4f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '600f1bbf62b2434ba1c0265c40017e63'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:44.338889 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-de89a9dc-2bd0-448e-9a6d-473e0426c4f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '600f1bbf62b2434ba1c0265c40017e63'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:44.352743 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-de89a9dc-2bd0-448e-9a6d-473e0426c4f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '600f1bbf62b2434ba1c0265c40017e63'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:44.353578 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-de89a9dc-2bd0-448e-9a6d-473e0426c4f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '600f1bbf62b2434ba1c0265c40017e63'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:44.368924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1420/2851] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:12:44 2022] DELETE /identity/v3/users/600f1bbf62b2434ba1c0265c40017e63 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:12:44.544941 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5216f-db3a-4251-8a29-a98a9a2883d9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:44.546384 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a8e5216f-db3a-4251-8a29-a98a9a2883d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:44.559905 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5216f-db3a-4251-8a29-a98a9a2883d9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:44.578128 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:44.621007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92abd09-55e2-4e28-a09a-28c6cde5fe61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:44.621863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92abd09-55e2-4e28-a09a-28c6cde5fe61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:44.622147 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92abd09-55e2-4e28-a09a-28c6cde5fe61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:44.622424 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92abd09-55e2-4e28-a09a-28c6cde5fe61 admin admin] PATH_INFO: `/v3/projects/f9a0c68e37d24845be24ba44937102ad` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:44.623637 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92abd09-55e2-4e28-a09a-28c6cde5fe61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9a0c68e37d24845be24ba44937102ad)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:44.625557 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92abd09-55e2-4e28-a09a-28c6cde5fe61 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:44.676625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f92abd09-55e2-4e28-a09a-28c6cde5fe61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9a0c68e37d24845be24ba44937102ad', 'request_id': 'req-f92abd09-55e2-4e28-a09a-28c6cde5fe61'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:44.686106 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f92abd09-55e2-4e28-a09a-28c6cde5fe61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9a0c68e37d24845be24ba44937102ad', 'request_id': 'req-f92abd09-55e2-4e28-a09a-28c6cde5fe61'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:44.697307 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f92abd09-55e2-4e28-a09a-28c6cde5fe61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9a0c68e37d24845be24ba44937102ad', 'request_id': 'req-f92abd09-55e2-4e28-a09a-28c6cde5fe61'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:44.709876 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f92abd09-55e2-4e28-a09a-28c6cde5fe61 admin admin] The token cache is being invalidate because project f9a0c68e37d24845be24ba44937102ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:12:44.710166 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f92abd09-55e2-4e28-a09a-28c6cde5fe61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:44.710862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f92abd09-55e2-4e28-a09a-28c6cde5fe61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:44.712770 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1432/2852] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:12:44 2022] DELETE /identity/v3/projects/f9a0c68e37d24845be24ba44937102ad => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:44.718781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf6668-f71b-449f-8b20-71fe638c9cd0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:44.719280 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf6668-f71b-449f-8b20-71fe638c9cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:44.719789 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf6668-f71b-449f-8b20-71fe638c9cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:44.746474 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-80bf6668-f71b-449f-8b20-71fe638c9cd0 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:44.748296 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-80bf6668-f71b-449f-8b20-71fe638c9cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:44.821472 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1421/2853] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:44.829304 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6d7b8-3f68-422c-a70a-d08698ade246 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:44.829591 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6d7b8-3f68-422c-a70a-d08698ade246 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:44.829894 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6d7b8-3f68-422c-a70a-d08698ade246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:44.848268 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4cd6d7b8-3f68-422c-a70a-d08698ade246 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:44.849576 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4cd6d7b8-3f68-422c-a70a-d08698ade246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:44.859366 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1433/2854] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:44.864034 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf6668-f71b-449f-8b20-71fe638c9cd0 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:44.875029 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf6668-f71b-449f-8b20-71fe638c9cd0 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:44.879583 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641b065-8904-4ea4-95a4-31b6a4b53fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:44.880222 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641b065-8904-4ea4-95a4-31b6a4b53fdb admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:44.880503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641b065-8904-4ea4-95a4-31b6a4b53fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:44.880775 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641b065-8904-4ea4-95a4-31b6a4b53fdb admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:44.881490 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641b065-8904-4ea4-95a4-31b6a4b53fdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:12:44.881790 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641b065-8904-4ea4-95a4-31b6a4b53fdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:44.883398 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641b065-8904-4ea4-95a4-31b6a4b53fdb admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:44.894924 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1422/2855] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:12:44.899354 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6d7b8-3f68-422c-a70a-d08698ade246 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:44.909006 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6d7b8-3f68-422c-a70a-d08698ade246 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:44.913719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc1993-5431-4411-a85f-9adeb2e03c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:44.914367 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc1993-5431-4411-a85f-9adeb2e03c31 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:44.914687 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc1993-5431-4411-a85f-9adeb2e03c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:44.915588 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc1993-5431-4411-a85f-9adeb2e03c31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:44.916372 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc1993-5431-4411-a85f-9adeb2e03c31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:44.918286 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc1993-5431-4411-a85f-9adeb2e03c31 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:44.944134 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1434/2856] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:12:44 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:12:44.948503 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641b065-8904-4ea4-95a4-31b6a4b53fdb admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:44.958711 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641b065-8904-4ea4-95a4-31b6a4b53fdb admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:45.015428 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01a891-f743-4350-a722-5571f18fbcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:45.016008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01a891-f743-4350-a722-5571f18fbcdc admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:45.016285 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01a891-f743-4350-a722-5571f18fbcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:45.016553 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01a891-f743-4350-a722-5571f18fbcdc admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:45.017224 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01a891-f743-4350-a722-5571f18fbcdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:45.019154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01a891-f743-4350-a722-5571f18fbcdc admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:45.038959 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1423/2857] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:12:44 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:12:45.043693 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc1993-5431-4411-a85f-9adeb2e03c31 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:45.053855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc1993-5431-4411-a85f-9adeb2e03c31 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:45.059768 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82c539-4916-4552-a197-cdb577b05f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:45.060429 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82c539-4916-4552-a197-cdb577b05f05 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:45.060719 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82c539-4916-4552-a197-cdb577b05f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:45.060987 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82c539-4916-4552-a197-cdb577b05f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:45.061709 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd82c539-4916-4552-a197-cdb577b05f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:45.063338 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd82c539-4916-4552-a197-cdb577b05f05 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:45.074142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1435/2858] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:12:45.078068 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01a891-f743-4350-a722-5571f18fbcdc admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:45.088947 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01a891-f743-4350-a722-5571f18fbcdc admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:45.094186 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d854cd-5cb2-4924-b05e-7cd71370bc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:45.095182 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d854cd-5cb2-4924-b05e-7cd71370bc39 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:45.095462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d854cd-5cb2-4924-b05e-7cd71370bc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:45.095734 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d854cd-5cb2-4924-b05e-7cd71370bc39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:45.096560 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d854cd-5cb2-4924-b05e-7cd71370bc39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:45.098146 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d854cd-5cb2-4924-b05e-7cd71370bc39 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:45.110007 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1424/2859] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:12:45.114264 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82c539-4916-4552-a197-cdb577b05f05 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:45.124337 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82c539-4916-4552-a197-cdb577b05f05 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:45.128910 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031294d-2925-428b-9f66-ff53d54ce1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:45.129771 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031294d-2925-428b-9f66-ff53d54ce1f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:45.130052 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031294d-2925-428b-9f66-ff53d54ce1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:45.130327 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031294d-2925-428b-9f66-ff53d54ce1f3 admin admin] PATH_INFO: `/v3/projects/e085da84105246cdad4d1d4235f13d16/users/d7907da156f1472f9c837c5659d67450/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:45.145076 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031294d-2925-428b-9f66-ff53d54ce1f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e085da84105246cdad4d1d4235f13d16, user_id=d7907da156f1472f9c837c5659d67450, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:45.147754 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031294d-2925-428b-9f66-ff53d54ce1f3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:45.161091 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1436/2860] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:12:45 2022] PUT /identity/v3/projects/e085da84105246cdad4d1d4235f13d16/users/d7907da156f1472f9c837c5659d67450/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:12:45.172158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56f5e8-c8e4-4ca2-82a3-0911266a45b7 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:45.172453 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56f5e8-c8e4-4ca2-82a3-0911266a45b7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:45.172738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56f5e8-c8e4-4ca2-82a3-0911266a45b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:45.180453 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:45.202894 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ca56f5e8-c8e4-4ca2-82a3-0911266a45b7 None None] MFA Rules not processed for user `d7907da156f1472f9c837c5659d67450`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:45.204089 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ca56f5e8-c8e4-4ca2-82a3-0911266a45b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:45.267807 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1425/2861] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:45.272186 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031294d-2925-428b-9f66-ff53d54ce1f3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:45.322472 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:45.340376 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031294d-2925-428b-9f66-ff53d54ce1f3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:45.345037 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b03574-2ab7-4cdf-b435-445951ffa4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:45.345700 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b03574-2ab7-4cdf-b435-445951ffa4bd admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:45.345984 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b03574-2ab7-4cdf-b435-445951ffa4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:45.346263 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b03574-2ab7-4cdf-b435-445951ffa4bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:45.347018 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b03574-2ab7-4cdf-b435-445951ffa4bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:45.348907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b03574-2ab7-4cdf-b435-445951ffa4bd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:45.377099 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1437/2862] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:12:45 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:12:45.379438 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56f5e8-c8e4-4ca2-82a3-0911266a45b7 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:45.390614 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56f5e8-c8e4-4ca2-82a3-0911266a45b7 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:45.450035 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a344e8-870d-48ea-8b35-35529b167605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:45.450745 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a344e8-870d-48ea-8b35-35529b167605 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:45.451032 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a344e8-870d-48ea-8b35-35529b167605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:45.451313 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a344e8-870d-48ea-8b35-35529b167605 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:45.452010 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a344e8-870d-48ea-8b35-35529b167605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:45.453932 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a344e8-870d-48ea-8b35-35529b167605 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:45.473505 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1426/2863] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:12:45 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:12:45.478208 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b03574-2ab7-4cdf-b435-445951ffa4bd admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:45.488397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b03574-2ab7-4cdf-b435-445951ffa4bd admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:45.493100 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70a562-274c-4538-b39a-44da0781a1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:45.494320 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70a562-274c-4538-b39a-44da0781a1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:45.494625 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70a562-274c-4538-b39a-44da0781a1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:45.494909 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70a562-274c-4538-b39a-44da0781a1c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:45.495632 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70a562-274c-4538-b39a-44da0781a1c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:45.497223 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70a562-274c-4538-b39a-44da0781a1c4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:45.507697 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1438/2864] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:12:45.512987 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a344e8-870d-48ea-8b35-35529b167605 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:45.523172 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a344e8-870d-48ea-8b35-35529b167605 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:45.528417 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3494bc-7243-413d-af38-772bbd0141a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:45.529595 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3494bc-7243-413d-af38-772bbd0141a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:45.529882 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3494bc-7243-413d-af38-772bbd0141a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:45.530161 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3494bc-7243-413d-af38-772bbd0141a9 admin admin] PATH_INFO: `/v3/projects/7ea058a613694c949fb4238f5f8a64ca/users/5b25d236f49f4489b29067a086f98cfe/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:45.544861 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3494bc-7243-413d-af38-772bbd0141a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea058a613694c949fb4238f5f8a64ca, user_id=5b25d236f49f4489b29067a086f98cfe, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:45.547561 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3494bc-7243-413d-af38-772bbd0141a9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:45.560175 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1427/2865] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:12:45 2022] PUT /identity/v3/projects/7ea058a613694c949fb4238f5f8a64ca/users/5b25d236f49f4489b29067a086f98cfe/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:45.564207 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70a562-274c-4538-b39a-44da0781a1c4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:45.575907 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70a562-274c-4538-b39a-44da0781a1c4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:45.635263 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad183504-92cd-421d-aea9-78387f640204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:45.635927 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad183504-92cd-421d-aea9-78387f640204 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:45.636273 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad183504-92cd-421d-aea9-78387f640204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:45.636554 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad183504-92cd-421d-aea9-78387f640204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:45.637274 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad183504-92cd-421d-aea9-78387f640204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:45.638877 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad183504-92cd-421d-aea9-78387f640204 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:45.649007 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1439/2866] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:12:45.653895 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3494bc-7243-413d-af38-772bbd0141a9 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:45.664548 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3494bc-7243-413d-af38-772bbd0141a9 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:45.669319 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fd4d8-9d25-4d7d-9933-49daa19764c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:45.670196 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fd4d8-9d25-4d7d-9933-49daa19764c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:45.670509 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fd4d8-9d25-4d7d-9933-49daa19764c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:45.670823 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fd4d8-9d25-4d7d-9933-49daa19764c7 admin admin] PATH_INFO: `/v3/projects/7ea058a613694c949fb4238f5f8a64ca/users/5b25d236f49f4489b29067a086f98cfe/roles/e0ab182b79ab44f592864af0ad8507c5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:45.672937 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72fd4d8-9d25-4d7d-9933-49daa19764c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea058a613694c949fb4238f5f8a64ca, user_id=5b25d236f49f4489b29067a086f98cfe, role_id=e0ab182b79ab44f592864af0ad8507c5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:45.675617 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72fd4d8-9d25-4d7d-9933-49daa19764c7 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:45.685982 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1428/2867] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:12:45 2022] PUT /identity/v3/projects/7ea058a613694c949fb4238f5f8a64ca/users/5b25d236f49f4489b29067a086f98cfe/roles/e0ab182b79ab44f592864af0ad8507c5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:45.748167 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1377034-447c-47b5-b597-3584357acd37 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:45.748462 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1377034-447c-47b5-b597-3584357acd37 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:45.748748 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1377034-447c-47b5-b597-3584357acd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:45.777350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c1377034-447c-47b5-b597-3584357acd37 None None] MFA Rules not processed for user `5b25d236f49f4489b29067a086f98cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:45.778485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c1377034-447c-47b5-b597-3584357acd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:45.846884 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1440/2868] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:45.857787 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fd4d8-9d25-4d7d-9933-49daa19764c7 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:45.861595 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d72fd4d8-9d25-4d7d-9933-49daa19764c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:45.920003 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:45.926325 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fd4d8-9d25-4d7d-9933-49daa19764c7 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:45.931028 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4529f-7786-428e-a0ac-2d05d0c5ee7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:45.932197 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4529f-7786-428e-a0ac-2d05d0c5ee7b service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:45.932437 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4529f-7786-428e-a0ac-2d05d0c5ee7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:45.932720 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4529f-7786-428e-a0ac-2d05d0c5ee7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:45.996538 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4529f-7786-428e-a0ac-2d05d0c5ee7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:45.997966 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4529f-7786-428e-a0ac-2d05d0c5ee7b service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:46.006738 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1429/2869] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:12:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:12:50.197190 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1377034-447c-47b5-b597-3584357acd37 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:50.198678 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c1377034-447c-47b5-b597-3584357acd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:50.210474 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1377034-447c-47b5-b597-3584357acd37 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:50.215112 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb0442-5c33-4962-a635-6c54f95e296b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:50.215752 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb0442-5c33-4962-a635-6c54f95e296b service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:50.216036 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb0442-5c33-4962-a635-6c54f95e296b service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:50.216315 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb0442-5c33-4962-a635-6c54f95e296b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:50.225152 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb0442-5c33-4962-a635-6c54f95e296b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:50.226592 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb0442-5c33-4962-a635-6c54f95e296b service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:50.237159 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1441/2870] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:12:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:12:55.086890 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4529f-7786-428e-a0ac-2d05d0c5ee7b service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:55.149825 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4529f-7786-428e-a0ac-2d05d0c5ee7b service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:55.154708 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d204279-2e3d-438c-9e14-9a869eac4142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:55.155596 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d204279-2e3d-438c-9e14-9a869eac4142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:55.155880 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d204279-2e3d-438c-9e14-9a869eac4142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:55.156158 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d204279-2e3d-438c-9e14-9a869eac4142 admin admin] PATH_INFO: `/v3/users/d7907da156f1472f9c837c5659d67450` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:55.157342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d204279-2e3d-438c-9e14-9a869eac4142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7907da156f1472f9c837c5659d67450)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:55.159297 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d204279-2e3d-438c-9e14-9a869eac4142 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:55.206499 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3d204279-2e3d-438c-9e14-9a869eac4142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7907da156f1472f9c837c5659d67450'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:55.207095 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3d204279-2e3d-438c-9e14-9a869eac4142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7907da156f1472f9c837c5659d67450'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:55.215458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3d204279-2e3d-438c-9e14-9a869eac4142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7907da156f1472f9c837c5659d67450'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:55.228194 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3d204279-2e3d-438c-9e14-9a869eac4142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7907da156f1472f9c837c5659d67450'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:55.228844 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3d204279-2e3d-438c-9e14-9a869eac4142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7907da156f1472f9c837c5659d67450'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:55.241961 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1430/2871] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:12:55 2022] DELETE /identity/v3/users/d7907da156f1472f9c837c5659d67450 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:55.246696 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b4b49-b7d5-4143-8e68-8ac6c86d4663 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:55.247253 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b4b49-b7d5-4143-8e68-8ac6c86d4663 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:55.247404 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b4b49-b7d5-4143-8e68-8ac6c86d4663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:55.265948 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-370b4b49-b7d5-4143-8e68-8ac6c86d4663 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:55.267163 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-370b4b49-b7d5-4143-8e68-8ac6c86d4663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:55.293578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:55.330397 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1442/2872] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:55.340698 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d204279-2e3d-438c-9e14-9a869eac4142 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:55.342112 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3d204279-2e3d-438c-9e14-9a869eac4142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:55.364854 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:55.402012 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d204279-2e3d-438c-9e14-9a869eac4142 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:55.405956 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bc04d-5f88-4776-b04c-a24e1bfbce4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:55.406622 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bc04d-5f88-4776-b04c-a24e1bfbce4b service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:55.406910 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bc04d-5f88-4776-b04c-a24e1bfbce4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:55.407180 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bc04d-5f88-4776-b04c-a24e1bfbce4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:55.416796 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bc04d-5f88-4776-b04c-a24e1bfbce4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:55.418221 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bc04d-5f88-4776-b04c-a24e1bfbce4b service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:55.427997 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1431/2873] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:12:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:12:55.696798 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b4b49-b7d5-4143-8e68-8ac6c86d4663 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:55.698351 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-370b4b49-b7d5-4143-8e68-8ac6c86d4663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:55.712310 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b4b49-b7d5-4143-8e68-8ac6c86d4663 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:55.716970 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731fc9e4-7f04-4e23-bd40-6bd650aec695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:55.717832 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fc9e4-7f04-4e23-bd40-6bd650aec695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:55.718119 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fc9e4-7f04-4e23-bd40-6bd650aec695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:55.719027 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fc9e4-7f04-4e23-bd40-6bd650aec695 admin admin] PATH_INFO: `/v3/projects/e085da84105246cdad4d1d4235f13d16` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:55.720410 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731fc9e4-7f04-4e23-bd40-6bd650aec695 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e085da84105246cdad4d1d4235f13d16)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:55.722418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731fc9e4-7f04-4e23-bd40-6bd650aec695 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:55.772940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-731fc9e4-7f04-4e23-bd40-6bd650aec695 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e085da84105246cdad4d1d4235f13d16', 'request_id': 'req-731fc9e4-7f04-4e23-bd40-6bd650aec695'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:55.780701 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-731fc9e4-7f04-4e23-bd40-6bd650aec695 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e085da84105246cdad4d1d4235f13d16', 'request_id': 'req-731fc9e4-7f04-4e23-bd40-6bd650aec695'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:55.786970 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-731fc9e4-7f04-4e23-bd40-6bd650aec695 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e085da84105246cdad4d1d4235f13d16', 'request_id': 'req-731fc9e4-7f04-4e23-bd40-6bd650aec695'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:55.797773 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-731fc9e4-7f04-4e23-bd40-6bd650aec695 admin admin] The token cache is being invalidate because project e085da84105246cdad4d1d4235f13d16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:12:55.798065 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-731fc9e4-7f04-4e23-bd40-6bd650aec695 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:55.798765 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-731fc9e4-7f04-4e23-bd40-6bd650aec695 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:55.800520 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1443/2874] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:12:55 2022] DELETE /identity/v3/projects/e085da84105246cdad4d1d4235f13d16 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:12:55.804273 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bc04d-5f88-4776-b04c-a24e1bfbce4b service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:55.814414 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bc04d-5f88-4776-b04c-a24e1bfbce4b service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:55.873878 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774b2dc-df24-4954-9e5f-d8023b6f3679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:55.874785 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774b2dc-df24-4954-9e5f-d8023b6f3679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:55.875089 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774b2dc-df24-4954-9e5f-d8023b6f3679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:55.875360 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774b2dc-df24-4954-9e5f-d8023b6f3679 admin admin] PATH_INFO: `/v3/users/5b25d236f49f4489b29067a086f98cfe` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:55.876552 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774b2dc-df24-4954-9e5f-d8023b6f3679 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b25d236f49f4489b29067a086f98cfe)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:55.878466 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774b2dc-df24-4954-9e5f-d8023b6f3679 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:55.926248 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e774b2dc-df24-4954-9e5f-d8023b6f3679 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b25d236f49f4489b29067a086f98cfe'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:55.926918 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e774b2dc-df24-4954-9e5f-d8023b6f3679 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b25d236f49f4489b29067a086f98cfe'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:55.934395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e774b2dc-df24-4954-9e5f-d8023b6f3679 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b25d236f49f4489b29067a086f98cfe'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:55.951243 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e774b2dc-df24-4954-9e5f-d8023b6f3679 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b25d236f49f4489b29067a086f98cfe'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:55.956295 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e774b2dc-df24-4954-9e5f-d8023b6f3679 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b25d236f49f4489b29067a086f98cfe'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:55.972734 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1432/2875] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:12:55 2022] DELETE /identity/v3/users/5b25d236f49f4489b29067a086f98cfe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:56.249869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731fc9e4-7f04-4e23-bd40-6bd650aec695 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:56.253331 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-731fc9e4-7f04-4e23-bd40-6bd650aec695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:56.271888 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731fc9e4-7f04-4e23-bd40-6bd650aec695 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:56.331239 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d53200-8ab7-45f3-8d02-557285990621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:56.332027 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d53200-8ab7-45f3-8d02-557285990621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:56.332306 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d53200-8ab7-45f3-8d02-557285990621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:56.332578 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d53200-8ab7-45f3-8d02-557285990621 admin admin] PATH_INFO: `/v3/projects/7ea058a613694c949fb4238f5f8a64ca` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:56.333762 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d53200-8ab7-45f3-8d02-557285990621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ea058a613694c949fb4238f5f8a64ca)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:56.335685 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d53200-8ab7-45f3-8d02-557285990621 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:56.382528 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-43d53200-8ab7-45f3-8d02-557285990621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ea058a613694c949fb4238f5f8a64ca', 'request_id': 'req-43d53200-8ab7-45f3-8d02-557285990621'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:56.386648 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:56.391487 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-43d53200-8ab7-45f3-8d02-557285990621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ea058a613694c949fb4238f5f8a64ca', 'request_id': 'req-43d53200-8ab7-45f3-8d02-557285990621'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:56.399012 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-43d53200-8ab7-45f3-8d02-557285990621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ea058a613694c949fb4238f5f8a64ca', 'request_id': 'req-43d53200-8ab7-45f3-8d02-557285990621'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:56.410544 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-43d53200-8ab7-45f3-8d02-557285990621 admin admin] The token cache is being invalidate because project 7ea058a613694c949fb4238f5f8a64ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:12:56.410863 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-43d53200-8ab7-45f3-8d02-557285990621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:56.411506 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-43d53200-8ab7-45f3-8d02-557285990621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:56.413075 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1444/2876] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:12:56 2022] DELETE /identity/v3/projects/7ea058a613694c949fb4238f5f8a64ca => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:56.419030 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2555d-9f25-4769-ab55-4b73af4bbe18 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:56.419524 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2555d-9f25-4769-ab55-4b73af4bbe18 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:56.419653 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2555d-9f25-4769-ab55-4b73af4bbe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:56.436816 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-bfa2555d-9f25-4769-ab55-4b73af4bbe18 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:56.438013 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-bfa2555d-9f25-4769-ab55-4b73af4bbe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:56.477367 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:56.499321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1433/2877] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:56.507610 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719f61c-fd85-496e-9dd8-31cf952e099e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:56.507901 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719f61c-fd85-496e-9dd8-31cf952e099e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:56.508180 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719f61c-fd85-496e-9dd8-31cf952e099e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:56.527959 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6719f61c-fd85-496e-9dd8-31cf952e099e None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:56.529154 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6719f61c-fd85-496e-9dd8-31cf952e099e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:56.539142 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1445/2878] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:12:56.544581 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2555d-9f25-4769-ab55-4b73af4bbe18 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:56.559462 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2555d-9f25-4769-ab55-4b73af4bbe18 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:56.565113 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4aa87-f522-4f98-8069-3cc4d43a9403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:56.565771 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4aa87-f522-4f98-8069-3cc4d43a9403 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:56.566055 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4aa87-f522-4f98-8069-3cc4d43a9403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:56.566331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4aa87-f522-4f98-8069-3cc4d43a9403 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:56.567092 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4aa87-f522-4f98-8069-3cc4d43a9403 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:12:56.567400 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4aa87-f522-4f98-8069-3cc4d43a9403 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:56.569010 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4aa87-f522-4f98-8069-3cc4d43a9403 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:56.582250 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1434/2879] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:12:56.587542 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719f61c-fd85-496e-9dd8-31cf952e099e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:56.596986 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719f61c-fd85-496e-9dd8-31cf952e099e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:56.601639 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384118c3-0ab9-43f3-b1ba-f5c584bf9b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:56.602385 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384118c3-0ab9-43f3-b1ba-f5c584bf9b2d admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:56.602938 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384118c3-0ab9-43f3-b1ba-f5c584bf9b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:56.603233 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384118c3-0ab9-43f3-b1ba-f5c584bf9b2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:56.603938 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384118c3-0ab9-43f3-b1ba-f5c584bf9b2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:56.605874 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384118c3-0ab9-43f3-b1ba-f5c584bf9b2d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:56.631912 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1446/2880] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:12:56 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:12:56.636759 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4aa87-f522-4f98-8069-3cc4d43a9403 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:56.647094 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4aa87-f522-4f98-8069-3cc4d43a9403 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:56.705412 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f7fa7-40b2-4398-80a6-73cc95a1f52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:56.706079 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f7fa7-40b2-4398-80a6-73cc95a1f52f admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:56.706364 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f7fa7-40b2-4398-80a6-73cc95a1f52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:56.706672 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f7fa7-40b2-4398-80a6-73cc95a1f52f admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:56.707354 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f7fa7-40b2-4398-80a6-73cc95a1f52f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:56.709256 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f7fa7-40b2-4398-80a6-73cc95a1f52f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:56.729205 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1435/2881] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:12:56 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:12:56.733667 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384118c3-0ab9-43f3-b1ba-f5c584bf9b2d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:56.745246 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384118c3-0ab9-43f3-b1ba-f5c584bf9b2d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:56.749872 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049a231-dcd3-487a-b043-f4ef56bf07c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:56.750532 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049a231-dcd3-487a-b043-f4ef56bf07c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:56.750845 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049a231-dcd3-487a-b043-f4ef56bf07c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:56.751124 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049a231-dcd3-487a-b043-f4ef56bf07c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:56.751855 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049a231-dcd3-487a-b043-f4ef56bf07c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:56.753443 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049a231-dcd3-487a-b043-f4ef56bf07c8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:56.765296 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1447/2882] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:12:56.769525 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f7fa7-40b2-4398-80a6-73cc95a1f52f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:56.779814 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f7fa7-40b2-4398-80a6-73cc95a1f52f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:56.784355 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ce363-461a-4cbe-9479-cfa6fa615c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:56.785002 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ce363-461a-4cbe-9479-cfa6fa615c55 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:56.785281 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ce363-461a-4cbe-9479-cfa6fa615c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:56.785553 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ce363-461a-4cbe-9479-cfa6fa615c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:56.786259 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ce363-461a-4cbe-9479-cfa6fa615c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:56.787880 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ce363-461a-4cbe-9479-cfa6fa615c55 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:56.797843 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1436/2883] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:12:56.801949 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049a231-dcd3-487a-b043-f4ef56bf07c8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:56.812963 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049a231-dcd3-487a-b043-f4ef56bf07c8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:56.817538 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994c4fc-33a3-4d51-99fc-c80ca5a13aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:56.818440 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994c4fc-33a3-4d51-99fc-c80ca5a13aee admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:56.818777 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994c4fc-33a3-4d51-99fc-c80ca5a13aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:56.819051 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994c4fc-33a3-4d51-99fc-c80ca5a13aee admin admin] PATH_INFO: `/v3/projects/fa0ff581a424438e9712c9e6c1fee357/users/339b98845c6e4f3093fa0b61d0f855d5/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:56.834471 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b994c4fc-33a3-4d51-99fc-c80ca5a13aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa0ff581a424438e9712c9e6c1fee357, user_id=339b98845c6e4f3093fa0b61d0f855d5, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:56.837284 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b994c4fc-33a3-4d51-99fc-c80ca5a13aee admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:56.851649 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1448/2884] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:12:56 2022] PUT /identity/v3/projects/fa0ff581a424438e9712c9e6c1fee357/users/339b98845c6e4f3093fa0b61d0f855d5/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:12:56.868154 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096a2e1-077d-4700-a975-d7a0ebcbe87b None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:56.868573 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096a2e1-077d-4700-a975-d7a0ebcbe87b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:56.868849 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096a2e1-077d-4700-a975-d7a0ebcbe87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:56.895143 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1096a2e1-077d-4700-a975-d7a0ebcbe87b None None] MFA Rules not processed for user `339b98845c6e4f3093fa0b61d0f855d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:56.896330 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1096a2e1-077d-4700-a975-d7a0ebcbe87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:56.967361 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1437/2885] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:12:56 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:56.980731 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994c4fc-33a3-4d51-99fc-c80ca5a13aee admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:56.982453 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b994c4fc-33a3-4d51-99fc-c80ca5a13aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:57.068485 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994c4fc-33a3-4d51-99fc-c80ca5a13aee admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:57.073378 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f719cb-1f96-491c-a270-6a96b571f37e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8563a25d16134d639a2a8849976fed9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:57.074031 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f719cb-1f96-491c-a270-6a96b571f37e service glance] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:57.074317 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f719cb-1f96-491c-a270-6a96b571f37e service glance] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:57.074621 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f719cb-1f96-491c-a270-6a96b571f37e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:57.083517 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f719cb-1f96-491c-a270-6a96b571f37e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:57.085012 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f719cb-1f96-491c-a270-6a96b571f37e service glance] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:57.095680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1449/2886] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:12:56 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:13:06.964108 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:13:07.090161 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=68,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:13:25.887342 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096a2e1-077d-4700-a975-d7a0ebcbe87b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:13:25.952990 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096a2e1-077d-4700-a975-d7a0ebcbe87b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:13:25.957926 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7329dd-d104-468b-aa96-59e32274b7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:13:25.958793 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7329dd-d104-468b-aa96-59e32274b7cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:25.959318 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7329dd-d104-468b-aa96-59e32274b7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:25.959598 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7329dd-d104-468b-aa96-59e32274b7cd admin admin] PATH_INFO: `/v3/users/339b98845c6e4f3093fa0b61d0f855d5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:25.960906 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7329dd-d104-468b-aa96-59e32274b7cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=339b98845c6e4f3093fa0b61d0f855d5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:13:25.962939 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7329dd-d104-468b-aa96-59e32274b7cd admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:13:26.009912 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7b7329dd-d104-468b-aa96-59e32274b7cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '339b98845c6e4f3093fa0b61d0f855d5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:26.010812 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7b7329dd-d104-468b-aa96-59e32274b7cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '339b98845c6e4f3093fa0b61d0f855d5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:26.017949 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7b7329dd-d104-468b-aa96-59e32274b7cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '339b98845c6e4f3093fa0b61d0f855d5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:26.030329 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7b7329dd-d104-468b-aa96-59e32274b7cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '339b98845c6e4f3093fa0b61d0f855d5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:26.031006 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7b7329dd-d104-468b-aa96-59e32274b7cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '339b98845c6e4f3093fa0b61d0f855d5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:26.043839 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1438/2887] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:13:25 2022] DELETE /identity/v3/users/339b98845c6e4f3093fa0b61d0f855d5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:13:26.048334 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f08c6-0ffb-4996-aea6-7235181efe63 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:26.048834 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f08c6-0ffb-4996-aea6-7235181efe63 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:26.048946 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f08c6-0ffb-4996-aea6-7235181efe63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:26.066718 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c03f08c6-0ffb-4996-aea6-7235181efe63 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:13:26.067915 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c03f08c6-0ffb-4996-aea6-7235181efe63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:13:26.129368 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1450/2888] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:13:26.139531 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7329dd-d104-468b-aa96-59e32274b7cd admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:13:26.140926 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7b7329dd-d104-468b-aa96-59e32274b7cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:13:26.203022 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7329dd-d104-468b-aa96-59e32274b7cd admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:13:26.207189 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e5d92-9127-4661-97f8-3a924783d86a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:13:26.207945 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e5d92-9127-4661-97f8-3a924783d86a service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:26.208296 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e5d92-9127-4661-97f8-3a924783d86a service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:26.208571 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e5d92-9127-4661-97f8-3a924783d86a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:26.217698 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e5d92-9127-4661-97f8-3a924783d86a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:13:26.219155 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e5d92-9127-4661-97f8-3a924783d86a service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:13:26.229031 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1439/2889] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:13:26.502943 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f08c6-0ffb-4996-aea6-7235181efe63 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:13:26.504541 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c03f08c6-0ffb-4996-aea6-7235181efe63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:13:26.516599 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f08c6-0ffb-4996-aea6-7235181efe63 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:13:26.521210 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41f570-d0cc-429f-ad48-4113d823cdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:13:26.522121 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41f570-d0cc-429f-ad48-4113d823cdbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:26.527034 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41f570-d0cc-429f-ad48-4113d823cdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:26.527360 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41f570-d0cc-429f-ad48-4113d823cdbb admin admin] PATH_INFO: `/v3/projects/fa0ff581a424438e9712c9e6c1fee357` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:26.528685 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41f570-d0cc-429f-ad48-4113d823cdbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa0ff581a424438e9712c9e6c1fee357)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:13:26.530929 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41f570-d0cc-429f-ad48-4113d823cdbb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:13:26.583116 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0e41f570-d0cc-429f-ad48-4113d823cdbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa0ff581a424438e9712c9e6c1fee357', 'request_id': 'req-0e41f570-d0cc-429f-ad48-4113d823cdbb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:26.591325 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0e41f570-d0cc-429f-ad48-4113d823cdbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa0ff581a424438e9712c9e6c1fee357', 'request_id': 'req-0e41f570-d0cc-429f-ad48-4113d823cdbb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:26.597515 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0e41f570-d0cc-429f-ad48-4113d823cdbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa0ff581a424438e9712c9e6c1fee357', 'request_id': 'req-0e41f570-d0cc-429f-ad48-4113d823cdbb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:26.608921 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0e41f570-d0cc-429f-ad48-4113d823cdbb admin admin] The token cache is being invalidate because project fa0ff581a424438e9712c9e6c1fee357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:13:26.609213 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0e41f570-d0cc-429f-ad48-4113d823cdbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:26.609869 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0e41f570-d0cc-429f-ad48-4113d823cdbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:26.611461 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1451/2890] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:13:26 2022] DELETE /identity/v3/projects/fa0ff581a424438e9712c9e6c1fee357 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:13:26.616409 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c734e3-52eb-4f72-a0e5-2afacb3ca53d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:26.616727 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c734e3-52eb-4f72-a0e5-2afacb3ca53d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:26.617007 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c734e3-52eb-4f72-a0e5-2afacb3ca53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:26.637333 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-80c734e3-52eb-4f72-a0e5-2afacb3ca53d None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:13:26.638442 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-80c734e3-52eb-4f72-a0e5-2afacb3ca53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:13:26.655863 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:13:26.703574 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1440/2891] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:13:26.711418 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee1f96-143b-4b2e-bbe3-1e5c42040c38 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:26.711709 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee1f96-143b-4b2e-bbe3-1e5c42040c38 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:26.712537 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee1f96-143b-4b2e-bbe3-1e5c42040c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:26.732569 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-57ee1f96-143b-4b2e-bbe3-1e5c42040c38 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:13:26.733777 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-57ee1f96-143b-4b2e-bbe3-1e5c42040c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:13:26.741916 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1452/2892] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:13:26.746088 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c734e3-52eb-4f72-a0e5-2afacb3ca53d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:13:26.755663 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c734e3-52eb-4f72-a0e5-2afacb3ca53d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:13:26.760423 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adedfec-7de1-4d5d-9a5a-b6d4191417b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:13:26.761075 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adedfec-7de1-4d5d-9a5a-b6d4191417b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:26.761358 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adedfec-7de1-4d5d-9a5a-b6d4191417b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:26.761634 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adedfec-7de1-4d5d-9a5a-b6d4191417b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:26.762352 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adedfec-7de1-4d5d-9a5a-b6d4191417b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:13:26.762698 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adedfec-7de1-4d5d-9a5a-b6d4191417b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:13:26.764290 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adedfec-7de1-4d5d-9a5a-b6d4191417b8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:13:26.776268 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1441/2893] 10.222.0.70 () {58 vars in 1152 bytes} [Thu Jun 16 19:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:13:26.780850 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee1f96-143b-4b2e-bbe3-1e5c42040c38 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:13:26.789940 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee1f96-143b-4b2e-bbe3-1e5c42040c38 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:13:26.795227 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7352aea-b405-4c26-b483-4e6da560dc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:13:26.795870 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7352aea-b405-4c26-b483-4e6da560dc42 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:26.796150 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7352aea-b405-4c26-b483-4e6da560dc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:26.796421 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7352aea-b405-4c26-b483-4e6da560dc42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:26.797110 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7352aea-b405-4c26-b483-4e6da560dc42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:13:26.799040 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7352aea-b405-4c26-b483-4e6da560dc42 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:13:26.824214 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1453/2894] 10.222.0.70 () {60 vars in 1152 bytes} [Thu Jun 16 19:13:26 2022] POST /identity/v3/projects => generated 391 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:13:26.828220 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adedfec-7de1-4d5d-9a5a-b6d4191417b8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:13:26.838988 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adedfec-7de1-4d5d-9a5a-b6d4191417b8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:13:26.895275 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f2e87-f00c-453c-9a3c-4e5c695480b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:13:26.895961 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f2e87-f00c-453c-9a3c-4e5c695480b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:26.896130 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f2e87-f00c-453c-9a3c-4e5c695480b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:26.896410 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f2e87-f00c-453c-9a3c-4e5c695480b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:26.897121 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f2e87-f00c-453c-9a3c-4e5c695480b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:13:26.899049 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f2e87-f00c-453c-9a3c-4e5c695480b5 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:13:26.920700 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1442/2895] 10.222.0.70 () {60 vars in 1143 bytes} [Thu Jun 16 19:13:26 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:13:26.924738 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7352aea-b405-4c26-b483-4e6da560dc42 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:13:26.935286 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7352aea-b405-4c26-b483-4e6da560dc42 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:13:26.939945 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec81b18-ab67-4a43-824f-ed653309c60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:13:26.940604 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec81b18-ab67-4a43-824f-ed653309c60f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:26.940891 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec81b18-ab67-4a43-824f-ed653309c60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:26.941171 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec81b18-ab67-4a43-824f-ed653309c60f admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:26.941925 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec81b18-ab67-4a43-824f-ed653309c60f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:13:26.943680 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec81b18-ab67-4a43-824f-ed653309c60f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:13:26.954347 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1454/2896] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:13:26.958331 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f2e87-f00c-453c-9a3c-4e5c695480b5 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:13:26.969267 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f2e87-f00c-453c-9a3c-4e5c695480b5 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:13:26.974037 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3c7c3-0410-4df0-93bf-df1c555b56c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:13:26.974985 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3c7c3-0410-4df0-93bf-df1c555b56c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:26.975314 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3c7c3-0410-4df0-93bf-df1c555b56c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:26.975595 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3c7c3-0410-4df0-93bf-df1c555b56c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:26.976321 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3c7c3-0410-4df0-93bf-df1c555b56c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:13:26.978599 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3c7c3-0410-4df0-93bf-df1c555b56c2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:13:26.993972 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1443/2897] 10.222.0.70 () {58 vars in 1121 bytes} [Thu Jun 16 19:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:13:26.997949 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec81b18-ab67-4a43-824f-ed653309c60f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:13:27.008118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec81b18-ab67-4a43-824f-ed653309c60f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:13:27.013369 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dbd16-fc0f-4da8-b9db-96b38b096c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:13:27.014243 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dbd16-fc0f-4da8-b9db-96b38b096c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:27.014526 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dbd16-fc0f-4da8-b9db-96b38b096c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:27.014844 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dbd16-fc0f-4da8-b9db-96b38b096c57 admin admin] PATH_INFO: `/v3/projects/1991c9db71f042e8b2ce4103a3f6ac84/users/ea23a3058d46430b9454a45c933a10e1/roles/cf55e7c6398f45efaf10dec4fe153f8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:27.030358 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834dbd16-fc0f-4da8-b9db-96b38b096c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1991c9db71f042e8b2ce4103a3f6ac84, user_id=ea23a3058d46430b9454a45c933a10e1, role_id=cf55e7c6398f45efaf10dec4fe153f8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:13:27.033031 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834dbd16-fc0f-4da8-b9db-96b38b096c57 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:13:27.046237 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1455/2898] 10.222.0.70 () {60 vars in 1482 bytes} [Thu Jun 16 19:13:26 2022] PUT /identity/v3/projects/1991c9db71f042e8b2ce4103a3f6ac84/users/ea23a3058d46430b9454a45c933a10e1/roles/cf55e7c6398f45efaf10dec4fe153f8a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:13:27.057072 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f7423-4ff4-4ffe-b3a4-710906e07213 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:27.057361 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f7423-4ff4-4ffe-b3a4-710906e07213 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:27.057641 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f7423-4ff4-4ffe-b3a4-710906e07213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:27.084766 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c21f7423-4ff4-4ffe-b3a4-710906e07213 None None] MFA Rules not processed for user `ea23a3058d46430b9454a45c933a10e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:13:27.085977 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c21f7423-4ff4-4ffe-b3a4-710906e07213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:13:27.154833 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1444/2899] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:13:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:13:27.165350 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dbd16-fc0f-4da8-b9db-96b38b096c57 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:13:27.167280 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-834dbd16-fc0f-4da8-b9db-96b38b096c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:13:27.189173 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:13:27.228920 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dbd16-fc0f-4da8-b9db-96b38b096c57 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:13:27.232572 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8e8cd-4119-4df4-9cd5-4b6f2a926767 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47de73e3f3b4979bff83140bf287275', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:13:27.233217 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8e8cd-4119-4df4-9cd5-4b6f2a926767 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:27.233501 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8e8cd-4119-4df4-9cd5-4b6f2a926767 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:27.233779 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8e8cd-4119-4df4-9cd5-4b6f2a926767 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:27.242113 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8e8cd-4119-4df4-9cd5-4b6f2a926767 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:13:27.244118 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8e8cd-4119-4df4-9cd5-4b6f2a926767 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:13:27.253136 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1456/2900] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:13:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:13:37.151564 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:13:37.249096 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=17 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:13:45.485041 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f7423-4ff4-4ffe-b3a4-710906e07213 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:13:45.546458 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f7423-4ff4-4ffe-b3a4-710906e07213 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:13:45.551000 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c72fb5-fc1c-47e4-b485-0f2d0bf461bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:13:45.551862 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c72fb5-fc1c-47e4-b485-0f2d0bf461bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:45.552140 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c72fb5-fc1c-47e4-b485-0f2d0bf461bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:45.552408 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c72fb5-fc1c-47e4-b485-0f2d0bf461bf admin admin] PATH_INFO: `/v3/users/ea23a3058d46430b9454a45c933a10e1` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:45.553582 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c72fb5-fc1c-47e4-b485-0f2d0bf461bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea23a3058d46430b9454a45c933a10e1)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:13:45.556026 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c72fb5-fc1c-47e4-b485-0f2d0bf461bf admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:13:45.603781 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a7c72fb5-fc1c-47e4-b485-0f2d0bf461bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea23a3058d46430b9454a45c933a10e1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:45.604477 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a7c72fb5-fc1c-47e4-b485-0f2d0bf461bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea23a3058d46430b9454a45c933a10e1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:45.612008 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a7c72fb5-fc1c-47e4-b485-0f2d0bf461bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea23a3058d46430b9454a45c933a10e1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:45.627263 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a7c72fb5-fc1c-47e4-b485-0f2d0bf461bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea23a3058d46430b9454a45c933a10e1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:45.627974 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a7c72fb5-fc1c-47e4-b485-0f2d0bf461bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea23a3058d46430b9454a45c933a10e1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:45.644255 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1445/2901] 10.222.0.70 () {58 vars in 1223 bytes} [Thu Jun 16 19:13:45 2022] DELETE /identity/v3/users/ea23a3058d46430b9454a45c933a10e1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:13:45.649862 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832b9b0-0e35-473c-a396-9d750384ecb8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:45.650500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832b9b0-0e35-473c-a396-9d750384ecb8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:45.650500 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832b9b0-0e35-473c-a396-9d750384ecb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:45.668598 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8832b9b0-0e35-473c-a396-9d750384ecb8 None None] MFA Rules not processed for user `7e2576df31cd4791832f5441b6f98ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:13:45.669800 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8832b9b0-0e35-473c-a396-9d750384ecb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:13:45.734523 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1457/2902] 10.222.0.70 () {58 vars in 957 bytes} [Thu Jun 16 19:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:13:45.745108 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c72fb5-fc1c-47e4-b485-0f2d0bf461bf admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:13:45.746521 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a7c72fb5-fc1c-47e4-b485-0f2d0bf461bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:13:45.804964 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c72fb5-fc1c-47e4-b485-0f2d0bf461bf admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:13:45.809224 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbd967-63f9-48e3-9bfe-19c51168b59d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a83f6e7aa448d6984a78914c1748cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:13:45.809877 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbd967-63f9-48e3-9bfe-19c51168b59d service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:45.810163 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbd967-63f9-48e3-9bfe-19c51168b59d service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:45.810444 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbd967-63f9-48e3-9bfe-19c51168b59d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:45.819683 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dbd967-63f9-48e3-9bfe-19c51168b59d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:13:45.822242 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dbd967-63f9-48e3-9bfe-19c51168b59d service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:13:45.831207 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1446/2903] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:13:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:13:46.117974 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832b9b0-0e35-473c-a396-9d750384ecb8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:13:46.119817 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8832b9b0-0e35-473c-a396-9d750384ecb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:13:46.131705 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832b9b0-0e35-473c-a396-9d750384ecb8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:13:46.136575 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cd87f-fde9-4b74-b268-7688fd0c666d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e2576df31cd4791832f5441b6f98ba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417f1f7349214e82b739864de0303f59', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:13:46.137440 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cd87f-fde9-4b74-b268-7688fd0c666d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:13:46.137718 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cd87f-fde9-4b74-b268-7688fd0c666d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:13:46.138099 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cd87f-fde9-4b74-b268-7688fd0c666d admin admin] PATH_INFO: `/v3/projects/1991c9db71f042e8b2ce4103a3f6ac84` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:13:46.139446 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cd87f-fde9-4b74-b268-7688fd0c666d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1991c9db71f042e8b2ce4103a3f6ac84)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:13:46.141336 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cd87f-fde9-4b74-b268-7688fd0c666d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:13:46.188989 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0f5cd87f-fde9-4b74-b268-7688fd0c666d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1991c9db71f042e8b2ce4103a3f6ac84', 'request_id': 'req-0f5cd87f-fde9-4b74-b268-7688fd0c666d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:46.196652 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0f5cd87f-fde9-4b74-b268-7688fd0c666d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1991c9db71f042e8b2ce4103a3f6ac84', 'request_id': 'req-0f5cd87f-fde9-4b74-b268-7688fd0c666d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:46.203107 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0f5cd87f-fde9-4b74-b268-7688fd0c666d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1991c9db71f042e8b2ce4103a3f6ac84', 'request_id': 'req-0f5cd87f-fde9-4b74-b268-7688fd0c666d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:46.214174 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0f5cd87f-fde9-4b74-b268-7688fd0c666d admin admin] The token cache is being invalidate because project 1991c9db71f042e8b2ce4103a3f6ac84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:13:46.214399 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0f5cd87f-fde9-4b74-b268-7688fd0c666d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:46.215063 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0f5cd87f-fde9-4b74-b268-7688fd0c666d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:13:46.216608 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1458/2904] 10.222.0.70 () {58 vars in 1232 bytes} [Thu Jun 16 19:13:46 2022] DELETE /identity/v3/projects/1991c9db71f042e8b2ce4103a3f6ac84 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:13:55.826903 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:13:56.210396 dv-cinder-846172-1-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:17:26.657992 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbd967-63f9-48e3-9bfe-19c51168b59d service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:17:26.659602 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a3dbd967-63f9-48e3-9bfe-19c51168b59d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:17:26.728802 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbd967-63f9-48e3-9bfe-19c51168b59d service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:17:26.786117 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51d72b-25fe-436b-8e9e-6674a186f981 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '880d0299c38c4eb897c669a512e599be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34185125e75d4aad8ad5382ab22ee8a2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:17:26.786777 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51d72b-25fe-436b-8e9e-6674a186f981 service placement] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:17:26.787056 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51d72b-25fe-436b-8e9e-6674a186f981 service placement] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:17:26.787326 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51d72b-25fe-436b-8e9e-6674a186f981 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:17:26.790720 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9e51d72b-25fe-436b-8e9e-6674a186f981 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:17:26.801743 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51d72b-25fe-436b-8e9e-6674a186f981 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:17:26.803607 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51d72b-25fe-436b-8e9e-6674a186f981 service placement] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:17:26.813395 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1447/2905] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Jun 16 19:17:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:17:36.808883 dv-cinder-846172-1-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=58 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}